With the altimeter, we are flying along a constant pressure surface, which can actually go up and down several thousand feet.
In a long flight from the subtropical troposphere to the polar one, whith large changes in ISA dev and surface pressure, you can go from 2,000 above to 2,000 below geometric or GPS altitude while maintaining FL 390.