The answer is not 104% of design speed, it is 104% of an arbitrary reference. In other words, the number you must remember is 104. We could have made it 269%.
It is funny how people can think that 867 degrees C is a real number, and forget that 0 degrees is 273 degrees above absolute zero, so the degrees are arbitrary.
We paint lines on gages to show the limts, the numbers are actually optional. Would you behave any differently if we labled the top of the green arc 10 and the bottom 5?
It is simply easier to remember 104% than it is 423 rpm, and easier to say.
The Russians actually set 100% at the max rpm in an auto, and cruise at about 82%. Go figure!