GrassStrip
You asked for opinions, so...
The vast majority of fresh PPLs, about 90-95%, chuck it in before their first renewal, and most do so with only 10-20 hours post-PPL. It is highly likely that you will be one of those. If you want to continue flying, then (assuming you've got the money) you need to minimise your workload and minimise your airspace busts to maximise your enjoyment.
Visual nav is indeed pretty hard, until so you have so many hours you can fly with your eyes shut (well almost). And even then it is often very hard to POSITIVELY identify a piece of forest or a lake among many others which look just the same. Don't let anyone tell you otherwise.
There appear to be two main sorts of private pilots:
Group A - These often got their license many years ago, fly only locally or to places they've been to many times, fly only on nice days. Not really interested in actually going somewhere for a purpose (other than a beer, on a nice day). These usually dislike GPS, often intensely. They don't need one because of the flying they do and/or because they know every shed under the routes they regularly fly. Occassionally they venture into bad weather, get lost, bring down half the D&D services, and sometimes they get squeezed between clouds and hills... They are frequently "uncertain of position" - the officially approved name for being lost.
Group B - These want to go to new and interesting places. They progress to instrument flying (you have to to actually go anywhere for real in the UK) and they use radio nav (VOR/DME) and GPS all together. These are the minority but they clock up the most hours, and very very rarely get lost.
If you go to one of the CAA safety seminars, it will be painfully obvious after about 10 mins which group the material is aimed at. Actually the last CAA man told me so...
GPS is perfectly OK for primary navigation. It is far more accurate, reliable and resistant to gross errors than anything else (including visual nav). You always need a backup of course, and visual nav is your backup in VMC. You use the TWO TOGETHER.
There are some clowns who reportedly use a GPS solely without planning the flight so they have no backup. They give GPS a bad name. But you've got to remember that if you fly into a hill and die and you used visual nav, that's officially permitted. If you flew a perfectly straight line into the same hill because you were tracking a GPS, that's not officially permitted and will be used for years afterwards as an example of how evil GPS is
However if you really think that a GPS costs too much then you are unlikely to ever acquire enough currency for any of this to matter. When I used to self fly hire the usual school junk, with decrepit avionics, the first things I bought were a GPS and a decent headset.