I'm fairly certain a UK drivers licence is NOT a valid document for travel from the UK to the rest of the EU.
As it happens on the majority of trips to Europe I have not had my passport checked, certainly I didn't coming into Brussels Charleroi last year. Also coming back into a non-designated airports in the UK having sent a GAR form I've never known customs/immigration actually turn up to check. Probably not worth the risk though because formally one should carry it in the UK's absence of a national identity card.