Because the IRSs will have trouble aligning with sufficient accuracy. They rely on sensing Earth rotation and establishing latitude. The nearer you get to the Poles, the more of a problem it becomes. If you consider a horizontal gyroscope at the equator, it will very quickly sense a tipping force as the Earth rotates. At the Poles, the tipping force will not be there, the Earth's rotation will be masked by the motion of the gyroscope wheel itself. But quite why the latitude limits are not the same North/South I have never understood, but they have designed the system to work at all likely terminals!