Lew
Not sure I'd enjoy flying with these mates of yours, I'm afraid. As several experienced guys on here have said, when flying with other pilots there should be some clear rules. Someone is always PIC and they are IN COMMAND during this period. No-one is allowed to touch controls or make flight decisions unless that PIC has allowed it and this other pilot has the requisite qualifications etc.
If they are not comfortable with this, they really should not fly with you, you either agree PIC per leg or they are fully PIC and you play PAX or safety pilot.
Frankly, these situations between responsible pilots should not occur. Two (or three) pilots in a plane should mean greater safety and awareness, not less, and greater risks/confusion. With the right guys, whatever the experience totals in the cockpit, you can work as a wonderful team. Sometimes it's instinctual, sometimes it requires very clear guidelines (never bad to confirm in any case...)
You learn pretty quick who you are comfortable to fly with and who you are not. This is a skill worth sniffing out pretty early on and sometimes you need to make frank assessments or clear these things up with flying pals.