Hi,
Almost 100 years ago, Sperry and others used the fact that the axis of a spinning gyro, with gravity causing precession, will align parallel with earth's spinning axis and hence they had something which will point to True North. They didn't need GPS to find True N. The Sperry Gyrocompass will point at True N. for ever even when at sea. It doesn't just "remember it" - it constantly finds it.
See the appendix in
Sperry Gyrocompass Mark 14
An IRS Laser gyro does a similar job. It can work out the direction of True North during the align process. It measure the rate of apparent wander and it's direction and then calculates it's Latitude (N or S) without any input from GPS or gate position. (For a gyro aligned parallel with earth's surface, Apparent wander is zero at the equator and 360 degs in 24 hours at the poles.) We have to put in present position because it can't sense Longitude (and it's a cross check that we know where we think we are against the Lat it accepts.)
The IRS constantly measures the direction of plumb vertical using accelerometers and hence can sense again the direction of True North. Once it's Navigating and moving over the earth's surface, it needs to know ground speed and position to allow for sensed errors (used to be called steaming errors on Ships with Sperry's gyroscope).
When in ATT, there is no navigation data so "steaming errors" are quite large, hence we have to update the direction frequently.