Helicopter pilots run into all this (no pun intended) quite often, since we frequently land somewhere other than a licensed airfield with a runway. There is definitely no exemption from the part of Rule 5 which refers to congested areas, and no precise definition of a congested area.
Balloonists also...
I was led to believe that the CAA regarded a "congested area" as anywhere marked in yellow on the half-million...
The only thing to do is play safe and not annoy anyone on the ground...
Absolutely
