Couple of possibilities:
1/ Your altimeter is measuring a pressure level, not necessarily altitude above mean sea level. IOW, 29.92 pressure level is not altitude.
2/ GPS errors are more apparent in altitude than laterally. This is a factor for GPS, but solved by WAAS or LASS.
GF