It's interesting to note that from posts here it appears that the #1 factor which limits currency is money.
In that respect this debate is no different from one about what level of landing fees is reasonable, etc.
However there must be a lot of people who do have the money but find that flying does not present a sufficient attraction. There are hassles with the weather and (if renting) with aircraft condition, availability, not being able to book one for long weekends, etc.
One generally has a fixed amount of dosh coming in, and one will spend it according to the "return". While there are no doubt people who are absolutely desperate to fly and will spend every penny on it (and who are probably over-represented in these online forums) most people will make choices. I bet many find that the "social scene" at their local flying set-up is a turn-off for both them and more so for their female companion! So the money gets spent doing something else.
I don't think 12hrs every 2 years is enough for flying from A to B in UK weather, but a lot of people don't do that. They pick a sunny day for a local bimble, perhaps with a passenger who is more current. It is true that such a pilot could legally go off in 3000m vis etc etc but most of them are smart enough not to. So this cannot be solved by regulation.