Put in another way: take a volume of air, e.g. a cylinder, that starts on the ground and goes all the way up to the end of the atmosphere, at ISA conditions.
Inside it, at an altitude of say 1000 ft, an altimeter will read exactly that value.
Now cool down the whole cylinder. The pressure at ground level will not change, because, as user ft points out, it depends exclusively on the total mass of air contained by the cylinder. The density, instead, can only increase, because P = ρ R T (if T goes down, ρ must go up, for P to stay the same; R is constant).
A higher density means that a bigger mass of air settles down below our 1000 ft level, compared to ISA conditions, therefore less air will sit on top of it, and the pressure at 1000 ft will be lower than ISA, i.e. the altimeter will over-read.
I hope it makes sense.
Deeday