I've only once been asked for a passport when flying privately between England and France. This was on leaving L2K - but it seems that even then you don't need a passport if you follow this protocol, which I have observed in action:
You: "Sorry mate, I left my passport in the aircraft, OK if I just go and get it for you?"
Policeman: "Yeah, sure."
You: Walk to your aircraft, get in and take off.