An interesting one a few weeks ago. I'd visited a local airfield (2 miles from my house) which is normally a gliding site on a flyaway from Perth. It's in EDI zone and the normal process is for VFR <2k, having to toggle between EDI App and the gliding site to negotiate clearances, etc.
On departure, I was asked by EDI to climb out, but remain within the airfield boundary at <2k. It was not possible to remain within the boundary, but at 500 ft on the climbout, I turned back to the airfield and orbited there, climbing to 1800' QFE. In doing so, I flew over the (*NEW - less than 3 yrs old!*) house of someone who I now know to be a NIMBY (despite the fact that the airfield has been there since WW2 and powered aircraft movements are normally less than 3 per day). He is around 400m in a virtual straight line off the end of the main runway. He subsequently complained about me - as far as I'm concerned, I was complying with an ATC instruction (and Rule 5 applied anyway), plus there is no 'formal' instruction to visiting pilots to avoid this particular housing estate.
IMHO, having to initiate any avoidance turns <500 ft AAL endangers the safety of the aircraft and as captain, that is my primary concern. I will comply as far as possible with local noise abatement procedures, but primary responsibility is to aviate; secondary concern is to navigate!!