It depends. Supplies differ in design, some are relatively crude and voltage varies with load. Others are regulated and voltage remains substantially constant. But a regulated supply on absolutely no load at all could read high. It depends what the design engineer (me) has decided to do.
You need to know what you are doing, you need to know what you are testing, and you need to apply appropriate loading to simulate the correct test condition.