How does it know whether to turn you left or right until you’re steering towards the target?
You have to look at it the opposite way. We are taught to navigate using, as one input to the puzzle, the direction we are pointing. The GPS, somewhat like a Martian, doesn't understand this cultural anomaly. It logically thinks you are really interested where you are actually going. So in common with the colour of the aircraft and the pressure in the tyres, it completely ignores orientation.
There are solid state chips nowadays that can detect the earth's magnetic field and others that can detect 'gravity' to give you orientation information. Unfortunately these seem to be always inferior to the traditional compass and gyro.