Peter,
It entirely depends on the GPS device. 'GPS' doesn't have a protocol for distances between waypoints, so it's at the discretion of the developer of the device. But as aviation navigation devices are designed for navigation rather than true vector/distance finding, then you'll have likely been exposed to more examples where the device measures waypoints as you describe.
The OP didn't state what the GPS device was, and as it sounded like it was an incredibly simple device, it may not have been programmed as a navigation device in the way you're used to.
So back to my original response 'it entirely depends in the device / settings'.
Also, the principal of GPS is actually incredibly simple. The translation of GPS into a navigation tool is extremely complicated. And of course, the implementation of GPS is an astonishing accomplishment in itself!