I am a flying instructor and would have concerns about some people doing it with others I wouldn't have an issue.
Then there is the magic day when my FI runs out. Do I magically over night loose the ability to allow some else to land while I am watching safely or for that matter fly in the RHS. In fact I haven't flown in the LHS of a SEP for getting on for 7 years now. I am sure that some schools of thought would require that I do some training before I can fly in the LHS again.
Its funny how the handling the controls causes great discussion but the none qualified pax doing the Navigation is never moaned about. Which in my view is by far the most important job while going from A to B.
We have 2 distinct areas here there is the giving a shot while at altitude which happens all the time, in fact its recommended if the pax is feeling air sick. And the landing of the aircraft/areo's/stalling/steep turns etc. Which still goes on but its not talked about as much but still legal.
There is under commercial ops restrictions on who can manipulate the controls but none in private ops.
There is a list of dutys in the ANO for the PIC, it doesn't include actually flying the machine.
The aircraft does define what flight crew you require onboard and its different depending on its certification sometime different depending on the reg.. Most POM state that one member of crew has to be at the flying controls at all times and for take off and landing all flight crew members must be at their assign stations.