Gasax, I'll be honest, the only place that argue over this is on pprune. I have been in touch with the CAA about this and they have stated that this:
the UK to fly below 500 ft provided they are 500 ft away from persons, vessels, vehicles and structures – in other words no change from the UK’s former '500ft Rule' that people flying in the UK are used to applying.
.. Is how we are to operate in the UK. So if you follow SERA, you're going to be safe and cautious, although we are allowed to operate under he previous 500' rule through the flexibility in the SERA document. This is where I think most of the confusion stems from.
It's actually laid down in black and white (well, blue) further in the main SERA document. Official Record Series 4 No.1065 General Permissions summarises it - pdf -
http://www.caa.co.uk/docs/33/ORS4No1065.pdf