If it came to it, I imagine that the magistrates would have to decide in a particular case. If the argument was about a low pass over a runway, all runways have something near them which could be deemed a structure even if the runway itself isn't (eg ILS aerials, fences, PAPIs, disused pillboxes etc).
A quick glance at legal definitions of structures includes:
"Anything constructed or erected with a fixed location on the ground, or attached to something having a fixed location on the ground including, but not limited to, buildings, walls, fences, swimming pools, radio towers, and porches, but not including driveways, paths, or other facilities, the sole purpose of which is to provide customary access to any structure."
"That which is built or constructed, an edifice or building of any kind, or any piece of work artificially built up or composed of parts jointed together in some definite manner, excluding fences in rear/side setback areas of a single family residence. Flood walls are considered to be structures."
"Any physical item built by a man or woman, from a shed to a high-rise or a bridge."
Notice that one includes fences, another specifically excludes them.
Personally I consider the 500' rule to apply to any land, but allow myself to fly as low as I like over the sea, being sure to avoid vessels by 500' (normally horizontally).