Without that exemption, instrument approaches on Runway 05 at Cambridge, for example, would be nigh on impossible. For those of us who live within the congested area on the final approach track, there is no comfort in Rule 5.
Most of the concern from the public about living under the 05 approach (which I get to hear about as a city councillor) is about an airliner crashing on them. I do explain to people that an EFATO in a light aircraft taking off from 23 is more likely. I hadn't thought about a SEP instrument approach to 05.