Which is why we state during a brief: "You have the airplane".
Somebody is always supposed to be flying, nothing new, old rules.
"Startle effect" has been mentioned.
IMHO if the former had been adopted the latter would not have happened. There would not have been any 'startle'. The PF, in tune with his instruments, would also have had a in tune 'seat if the pants' and there would have been a conflict of senses; and hopefully a WTF moment and a pause. This would have been true for both LHS & RHS. Easy to say from here, I know, but I think it might have prevented a rushed response. A good lesson to learn.