Flying as a passenger rarely, if ever, worries me. I spent enough time as an instructor to get used to others flying differently to how I might.
If another pilot is a passenger then it provides just a little more impetus for me to strive to do even better than the standard I usually set for myself.