I don't think it has much to do with whether flying is demanding or not. Once you have the licence you've probably the ability to cover the demanding bits.
It may well have something to do with the problem of access, whether for weather, booked planes, distance from airfield (I finally suspended my activities in 1971 when I was driving 1.3/4 hours to fly for 50 mins followed by 1 3/4 hour drive back.) Surely has something to do with the cost and commitment to family, GF etc (did for me too.) and a whole lot to do with the lack of help, encouragement, camaraderie, social aspect etc.
That surely is the really big change from the pre war and early post war flying clubs. They were also part of a social world, without TV, DVD, etc it people wet off to spend time with like minded friends. That world is now gone and the industry has not adapted or replaced it with anything worth while. Ten pin bowling has largely gone the same way, ad how many villages have Cricket clubs these days?
Everybody knows, ( or should) that it cost 10 times as much to get a new customer as it does to retain an old one, so why don't they spend a little on doing just that!
The new paradigm of 'Flight Centres" and 'Aviation Schools"doesn't encourage people to continue, they sell you the training and then boot you out into the wide world. Unfortunately most of those who went before have already dicovered how inhostpitable that world is and moved on so there is no supporting social ifrastructuure.
Pretty shortsighted.