Fly_For_Fun
It is very interesting that the main thrust of opinion about this is that the pilot in question should under no circumstances lose the privileges of his license.
No-one has either said or implied that. We don't know what 'the pilot in question' is alleged to have done or whether he actually did it, and know nothing about his flying history.
As a general comment, IMHO it is important to distinguish between low flying and endangering. Breaching Rule 5 is not necessarily endangering, just as exceeding the speed limit is not necessarily dangerous driving.
I suppose you should not take licensing action against a driver who drinks and drives or causes death through careless or dangerous driving.
I don't understand how you get to that from
"The CAA cannot and should not take licensing action in order to punish the licence holder. (Punishment is for the court to decide.)" 
Perhaps the courts should have powers to remove the privileges of a licence granted by the CAA.
I have seen no evidence whatsoever, during almost four decades in the courts and dealing with numerous aviation cases, which leads me to believe there is any need for the additional power you suggest. (Nor IMHO is it desirable.) Under the existing procedure, the CAA revokes or suspends a licence/certificate if a pilot's conduct demonstrates that he/she does not meet the necessary criteria to hold it.
What a strange attitude some pilots have toward the actions of their own kind, no matter what the misdemeanour.
I agree. Some pilots are far too quick to condemn fellow pilots, often suggesting that they should lose their licences for even the most trivial misdemeanour. A very strange attitude but, sadly, not uncommon.
it is the idea that GA can self regulate safely that is in question, and this seems to be born out by some of the opinions here
GA does not self regulate. With very few exceptions, it is regulated by the CAA. (Some would say too regulated.)
Which "opinions here"?
You make these sweeping comments about people's opinions and posts, but have consistently declined to point to any that support your contention.
I gained the impression from your posts in another thread that you appear to have (at best) mixed views about professional pilots who were not previously military pilots, and, as in this thread, a low opinion of private pilots. I have enormous respect for mil pilots and mil training and, when a choice was available, have always chosen to train with ex-QFIs/QHIs. That said, the mil way of flying is not always practical - or necessary - in the civilian world.
Regulations exist for the safety of the flyers and the public.
In
very broad terms, that's true. Whether all are actually necessary is a matter for another debate. However, IMHO it is neither necessary nor desirable to prosecute every breach regardless of the circumstances and regardless of how minor the breach..
With respect, I wonder if the 'problem' you see is more in your mind than real? I've no doubt pilots occasionally fail to strictly comply with the regulations, and it may well be that civilian pilots don't always adhere to the regs as strictly as mil pilots do in the mil world, but the reality is that UK aviation (both commercial and private) has an excellent safety record. IMHO the existing system works well and does not need to be changed.
that would suggest to me that the CAA needs to start discharging its duties correctly and fully
There are a few areas where some would say the CAA fails in its duty (
eg failing to investigate/prosecute illegal public transport, failing to fully understand and properly tackle the problem of laser attacks on aircraft, particularly helicopters) but, in my experience, the CAA generally discharges its duties correctly and fully.
Non-PC Plod
is it reasonable to infer you have to be pretty blatant/wilful/dangerous/extreme to attract the attention of their legal department?
No.
JimBall
Instead we live with a scenario that everyone, including the FOI of the CAA, hates the drafting of Rule 5, but does nothing about it.
Rule 5 is drafted by the CAA - as are all the Rules of the Air, and the ANO. Rule 5 was re-drafted only a few years ago. (I agree the FAA low flying rules for helicopters are much more sensible.)
John R81
I don't doubt your research of the law gave you pleasure but it's of no practical value. To the best of my knowledge, no-one has ever been prosecuted for breaching Rule 5 by flying too close to a fence nor, for practical evidential reasons, is anyone ever likely to be.
.