Not necessarily infringing if under SVFR.
I have flown under an SVFR in that area, routing from the South to White Waltham, following the M25 into the CTR, passing between Brooklands and Fairoaks, then via Thorpe interchange (M25 -M3) to Ascot racecourse, then WW.
I've also departed Fairoaks North to Thorpe, then Ascot, Burnham and out towards Bovingdon.
The clearance in both cases was not above 1000 feet. Obviously one must be mindful of the 500 foot and alight clear rules (if SEP). Plenty of green spaces, though.
FBW