My problem with QFE is one of traffic awareness. At the numerous mil airfields where there are adjacent civvy airfields (eg Brize/Oxford), having QNH set reduces a lot of the ballache about "if he's at 2000ft, what's he got set on his altimeter?". Clutch QFE is not a solution to this: the solution, as far as I'm concerned, is for everyone to use QNH and just realise that the ground isn't necessarily when the altimeter says 0ft.
The only exception to this is for a PAR. Even for ILSs, the rest of the world seem to manage very adequately with QNH approaches.
Flying on aerodrome QNH greatly reduces workload when it comes to coordination, situational awareness and can significantly help airspace infringements. All it requires is for people to realise that the ground isn't going to be at 0ft.
And, when one considers that most military pilot, when below 5000ft AGL can tell their height to +/-100ft just by looking at the ground features (isn't that part of the training?), I really can't understand why the RAF went back to using QFE after coming into line with the rest of the world.