There's no requirement to carry your pilot's licence for private flights within the UK, as far as i'm aware, so you only need to carry it if going abroad, in which case you'd need your passport anyway.
I don't carry my license when i go flying - have I been doing it wrong?!