I agree...I don't like this idea of 'pull ALT' as soon as you get the higher clearance. Obviously (as you say) besides the noise issue there's the fact that you might not make a critical altitude constraint due to accelerating low level.
There is another alternative.
Set the newly cleared altitude in the window and then PUSH ALT.
What do you think of that idea?