First of all, all GPSs work in an aviation environment. What makes an aviation GPS unique is the ability to hold a database with not just waypoints, but also airspace structures and various details about these. Plus vertical navigation, in some cases.
If you just want a "find your way home" GPS, you can just get any ol' outdoor GPS and put your home airport in there as a waypoint using its lat/long coordinates from the AIP/AFD/whatever (or simply press "Mark" when standing on the apron).
In fact, this is exactly what I'm using when I'm not navigating by map, compass and stopwatch. I've got a Garmin eTrex Euro (the cheapest, least capable of the eTrex series, but it was the only one available when I bought it over 10 years ago). It's manually filled with all the aviation waypoints that I could find in the area that's relevant for me (airports, VRPs, various beacons) and I simply use one of those as a "direct to" waypoint, or string a bunch of those together to create a route.
I did get the data cable and use OziExplorer with the Jeppesen VFR maps to make the process of loading waypoints and routes a bit easier though.
The eTrex Euro (or the US equivalent, which is exactly the same but has words instead of pictograms for the knobs) can be had new for well short of 100 USD, and second-hand for maybe 50 or so.
The standard caution about using GPSs as backup navigation tools does apply though: If your GPS has been living in your flight bag without being turned on regularly, or if you have moved a significant distance since it was last turned on, it may take as much as FIFTEEN MINUTES before you get a reliable fix. So, ironically, you will want to turn the GPS on before you get into trouble.
(And make sure you've got some spare batteries with you. The eTrex uses two AA batteries so just throw a pack of alkaline AAs in your flight bag and you'll be fine for the next ten years.)