Some good advice here. GPS is excellent if used wisely.
If you rely on your unapproved GPS, and get into "difficulties", you can be sure that will be the moment the GPS decides not to work.
There's a sadistic but wise CAA examiner round here. He does my IMC rating renewal every two years. He turns off the GNS430 and the ADF before we start a long session under the hood, doing unusual attitude recoveries and partial panel exercises. After that's done, I have no idea where I am. He then asks me to show him where on the map we are. This is still under the hood, but being visual doesn't always help over East Anglia.
With one VOR it's doable. With two VORs, it's easier. With a DME as well, it's easy. (But it only works if there are navaids in range.) That's a skill I think is worth practising.
If you can't "find" yourself without the GPS, then I'd say don't rely on the GPS as your only backup!