In my experience, hand flying, especially during departures, puts a heavy load on the PNF/PM. Everyone's SA is also reduced during hand flying.
An opposite point of view written by a Pprune contributor during another thread on automation was "automation robs situational awareness through absence of physical/sensory cues of flying the machine and as a result atrophies flying
and thinking skills.