I think the PIC would be in a lot more trouble to hand over the controls to a "non-rated" pilot / passenger in case of emergency / trouble than the "flying passenger".
But donīt worry - after the case of emergency you wonīt face any touble. But your left behind relatives would.
