The difference between the actual track and the nav track was, presumably, caused by the drift in the AINS.
If you move the actual track over to the previous nav track (ie, the yellow line), with the same amount of AINS drift, then the aircraft gets very close to Mt Bird, which is 1766 meters or 5790 feet. That's fine when he's at 10000 feet heading back to the north, but when he turns back towards the south and descends to 1500 feet, Mt Bird, which he thinks is to his left, would become important. Yet even after the F/O says it's clear to the right, he decides to go left, at an altitude of 1500 feet towards a 5790 foot mountain.
I think it was a case of "I think I'm here, but I might be there."