I've been flying so far with just a Garmin eTrex Euro in my flight bag. Using OziExplorer I entered all the coordinates of all Dutch and Belgium airfields as simple waypoints, extending that database as and when required. Most flights I do are purely visual, sometimes I turn that GPS on as a backup, depending on weather and flight/route complexity.
Not to say that you *have to* buy a cheapo outdoor GPS, but the top end of the range (Garmin 695) is currently well over 1800 UKP and it's simply not necessary. But I have to say that I like the idea of having a backup GPS at hand to help me if I get unsure of position somehow, and the eTrex works for me.
For UK flying, the NATS Aware tool is just about the best value for money you can get.
Oh, and don't forget to learn how to use the GPS that might be in the aircraft already. You paid for them already, best to use them if they're there anyway. Garmin does an excellent simulator for the GNS430/530 series and up.