Dear oh dear. It's not about whether the pax notice the difference or even how big one's balls are, it is about what is the safest course of action. Most pilots can fly manually if required to a competent standard and it seems doubtful whether the pax would notice the difference. However by flying manually we are exposing the pax to human frailty which tends to show itself when the weather is bad etc. Of course it depends on the technology, but in most modern helicopters the technology is far more reliable than the pilot. Why should the pax have to suffer increased risk during a night flight in bad weather just so that the pilots can demonstrate to each other what heroes they are?
Fly manually when there are adequate safety margins by all means (ie reasonable weather) but not when safety margins are at their minima.
Any SAR pilot auto-hovering over the ocean at 40' on a foggy night who says "never mind this phenomenally clever and robust inertial auto-hover system, I'm going to take it all out and fly manually" (just to show how clever I am) should be sacked immediately, and the same philosophy extends to any sort of flight near published limits for helicopters with modern and robust autopilot systems.