There is a technical argument that a pilot flying under an FAA license needs to carry an ID and that a passport is the only document available in the UK which satisfies this. I read this somewhere, not sure where...
It is a bit moot since the only place one is likely to get turned over (by a copper equipped with a special "how to check FAA pilots" briefing document) is France and there one will have the passport anyway.