My opinon is that this is one of those things that if you are trying to prove that you did not break the regulation, you probably lack the evidence you need to support your position. If you go over at 400 feet, and no body cares, you have no problem. If you go over at 700 feet, and someone makes a stink about it, you'll have a heck of a time actually proving that you were high enough.
Though the regulation is written with numbers (because that is the only way), the real intent is; don't fly so that people on the ground are angered or feel threatened by the proximity of the plane.
I had occasion decades ago, while trying to prevent what could have been 3 unseen drownings, to fly over a very crowded beach at 20 feet to get the required attention. Everything worked out fine, and apparently my action helped. I called in, and reported myself, reasoning that someone should, so I my as well be first, and get the basis for a defense in. Months later I happened upon the same inforcement staff member, who told me that not one low flying report had come in about me! Go figure... Did everyone on that beach, and all of the neighbouring cottages, know what I was trying to accomplish!
Just fly like a good neighbour as much as you can... Numbers are hard to prove in this instance...