I think the UK should re-define the definition to mean 'by sole reference to instruments',
You miss the point of IFR (and VFR). Flight rules exist to enable aircraft to avoid obstacles/terrain and other aircraft. They're not intended (by ICAO, EASA or anyone else) to dictate how the aircraft is controlled or navigated.
The Instrument Flight Rules are a set of rules that exist to assist aircraft to avoid obstacles/terrain and other aircraft if they are unable to do so visually. Sense dictates that authorities
insist that aircraft follow these rules if visual means are not possible, and
permit aircraft to follow these rules even if visual means are possible. A certain minimum set of met conditions (VFR minima) are set out as the boundary between when visual avoidance is and is not possible.
... So, flight 1000' above obstacles etc willl STILL be VFR if conducted in VMC.
I'd say 90% of my flying in the past 26 years has been IFR, most of it in VMC and not requiring me to exercise the privileges of any kind of instrument rating. Of course, it's been in Class 'G' airspace...
Just because it would have been possible for you to obey Instrument Flight Rules doesn't mean that you had to, nor that you were doing so intentionally. If you were also satisfying Visual Flight Rules, you probably intended to be a VFR flight.
Where the difference between the UK and other states comes is in situations where flight using visual means to control and navigate is possible, but the minima for VFR are not met. For example, that might be in a visibility of 4 km in an aircraft doing 160 knots, or it might be 500 ft above, or less than 1500 m from, scattered cloud above 3000 ft. Other states simply prohibit that for pilots without an IR. The UK offers the flexibility of permitting it, outside controlled airspace, provided the quadrantal rule and minimum altitude rule are met. Why do you want that flexibility taken away? It seems useful, and reasonable on safety grounds.