..... If you don't need really high resolution pictures, and "snap shots" will suffice, then you could look at some of the HP Ipaq PDA's ....
For example the hw6915 model ( Cost me £285 - 6 months ago ) in addition to being a Windows Mobile PDA, Mobile Phone, Wi-Fi, Bluetooth, etc etc, has built in GPS and a 1.3 megapixel camera.
When taking a picture, the GPS location will be saved with the picture ( Assuming of course that you are somewhere within range of a GPS signal ).