Why, what’s the problem with everyone working off the same datum?
They don't and even if, we use different reference frame over the globe.
Geometric altitude gets rid of many of the errors that plague pressure altimetry, like temperature, lapse rate, variability and not least, transcription. No need to know the surface pressure. What’s not to like?
My plane flies on pressure and air flow, not strict mathematical geometry. I prefer an altitude with a relationship to the physics of flying, not artificial rulers and dividers. If I get a QNH is does not only tell me about heights, it does tell me more on the things I have to expect when piloting - it connects to your guts feeling. I prefer an intuitional connection to the air outside.