I think its' true that for most people learning to fly is difficult, and often not exactly pleasant and often extremely frustrating. And it gets worse if you are "getting on a bit". The radio makes it much harder (due to brain overload) but one has to learn it sometime.
At some point is clicks though.
I started at about 43 and found it really hard; in fact I hated most of it. Flying solo, even within the PPL, was a fantastic liberator. But I still hated the cr*p aeroplanes I had to fly.
One could make it a lot easier and much more pleasant if one rented a better-than-normal plane, threw a bundle of £££ at a nice instructor (preferably an airline pilot, serving or retired) and told him to take you to interesting far away places (e.g. France or Spain) and you learn to fly gradually on the way. This is sometimes done in the USA, especially at the instrument rating level, made easier by the ability to get trained by freelance instructors (a much better system than the rigid JAA "flying school" thing). But this is a much more expensive option; probably 50% to 100% more expensive than going the standard intensive "45 hour" PPL, and most people learning to fly in the UK are verging on being skint, with many having only just enough to pay for the next lesson. So there is little demand for making the training more pleasant. Instructors would love it of course - they rarely get a chance to go anywhere.