From the point of view of gliding, I've done a fair bit of time looking at the back of the pupil's head, and also sitting next to (mainly first time) pupils in an old T21.
There are up and downsides to both systems. Side by side is great for reassurance of nervous early pupils and you can demo control movements, point at the relevant instruments etc more easily. Lookout on the pupil's side is a lot more difficult until you can trust the pupil to see what's on his side.
Looking at the back of the pupil's head is excellent for picking up poor lookout scans but basic demos may need more explanation. Sitting in front means that the pupil becomes accustomed to a single seat type cockpit and when the time comes to go solo the only real difference is the absence of nagging from the back seat!
Weight and balance is easier with tandem seating because the instructor is usually sitting nearly on the centre of gravity.
The military seem to change their minds on side-by-side or tandem instructing fairly often - a case of the grass being greener? Right now they seem to be in a tandem phase with the Tucanos and side by side with the Tutors.
Most gliders you don't have much choice these days - it'll be tandem which gives lower cross sectional area and better performance. Power is almost all side by side - because Cessna want to make you feel as if you're in a car?