Yep, for sure.
My instructors encouraged me to have the GPS on, as it's another pilot aid there in case I need it, etc. I always found a better sense of achievement by not using it though and actually seeing my destination on top of the nose cowling 25 mins out.
That said, none of the aircraft I fly with the company I'm with now have GPS in them anyway, so there's little choice.
520.