Maybe I am just weird but I never use the GPS. I guess I would turn it on if I got properly lost. Personally I really enjoy the navigation. It keeps you sharper and when that diversion comes into play then it's invaluable to know what your doing (imagine its a really bad day and the GPS also doesn't want to help you).
My favourite piece of equipment for diversions is the wind protractor. It's incredibly easy to use you obtain track and time very quickly corrected for the wind no less (provided you know the wind to begin with- which you should do!). I keep this little gem in my knee-board folder with two pencils both of which are marked at 10nm invervals for the chart scale I am using (that's a "just in case" thing and has bugger all to do with the wind protractor...obviously the pencils primary role is to make a dot). Its also fun to practice without any additonal equipment (except the chart!) - just to see how close I am.
I really recommend practicing diversions. It will save you such a headache when you have to do one for real. I routinely do these and the day I had to divert it wasn't a big deal. I have also delibertly flown to my routine alternate airports (as listed on my flight plan) just to ease the work load on arrival since it's much nicer to have a better idea what to look for/ know you surroundings in the event that you have to use them.
Polishing good nav skills is priceless (for eveything else there's.....).
I once heard on the radio: "My GPS failed. I am lost!". Mmm...
LG