Great points M.R and 3-Point.
The pre-flight briefing and deciding who does what is certainly a way to set the boundaries from the beginning as you rightly said 3 Point. I have often sat in CPL backseats and seen the Instructor tell the student to do the PAX brief and one of the items on the list is something along the lines of "Please refrain from interfering with the controls etc".
I have wondered though if it is a 2 seat aircraft and your buddy touches the controls, surely they wouldn't be stupid enough to "endanger" the safety of the flight...a) because they are a pilot also b) they are a higher license holder.. and if you firmly state you are PIC (on the ground) - that should nip it in the bud on the ground rather than in the air!
What about situations where pilot buddies in the RHS with good intention call out minimum height information in the air on approach..?
For example, you need to be "xxx feet crossing ABC"... or "you are xxx feet to high at ABC"... to which may elicit a similar response of "I am PIC and don't worry I wont let anything happen to you..or I will deal with it"..
I suspect it is probably always un-nerving if you haven't flown with this particular person before?
The explanation to which on the ground after tends to be ... This is what the co-pilot will do in a large commercial environment.. calling out "minimums". Whatever that means!
Scoobster