As BP says, one cannot really use a GPS as a get you out of trouble box because when you turn it on, it will take potentially ages to get a fix.
A GPS has to be running constantly throughout the flight.
If you want to "play traditional pilot" and not look at it, that's up to you.
That said, get yourself a GPS which runs an aviation moving map. The £150 Aware unit is pretty good. If your budget is bigger, get something with a bigger screen but which still runs the same "printed" VFR map.