As we all know the two instruments in IO's picture measure two completely different things.
The GPS is measuring the position of the aircraft, relative to the centre of mass of the earth (i.e. the thing the satellites are orbiting). The GPS then determines where it thinks the surface of the Earth is using a standard model Geoid. More sophisticated GPSs then take into account the local geoid undulation, or even the actual terrain data. The IFR GPS are pretty sharp at determining where they are in 3d, but the surface of the Earth may be in a slightly different place than the GPS thinks.
The altimeter measures the air pressure and converts the current pressure less the set pressure (the QNH) as a distance, assuming you are in a standard atmosphere. On a very cold day even only 5000 feet above the airfield that has provided the QNH, you could be 500 feet lower than the altimeter shows.
The altimeter will be very good for traffic separation as everyone sets the same reference point, but is less good for terrain avoidance on cold days.
A 'camping' GPS may have a very simple geoid model and hence not have an accurate local view of sea level (and hence get the wrong altitude).
However, like Fuji and IO have seen, once you do the temperature correction so you know what the altimeter meant to tell you! an IFR GPS (particularly one with TAWS) will give a very good true altitude