AF330
I think part of your mis understanding is the meaning of GPS PRIMARY.
It does not mean that GPS is number 1 and used to determine everything, but just that the GPS position going into the mix IRS/GPS position (GPIRS) complies with an integrity criteria - a reasonableness check.
If it does not comply, then GPS PRIMARY LOST is displayed.
When it again complies, then GPS PRIMARY is redisplayed.
Again, historically, aircraft had been certified without GPS and GPS was the new kid on the block - a bit like you! It was not 'trusted' and was used to supplement existing navigation methods, hence the crew retain to this day the ability to deselect GPS as an input to the navigation processes. Indeed the national governments that put up the satellites retained control over them and could 'modify' the positions they provided to deliberately create inaccuracies for their enemies - remember the satellite network was originally for the military, and you don't want your enemy to be able to use your system against you to accurately bomb targets!