ERJFO, I'd be intrigued to know what type you fly and who for as I find it hard to believe that neither your manufacturer or company do not publish advice on the matter. I'm assuming that you fly a commercial airliner.
Boeing Flight Crew Training Manual under Flap Usage sub heading Flap Maneuvering Speed Schedule.
"During Flap retraction/extension, selectingthe next flap setting should be initiated when reaching the maneuver speed for the existing flap position."
"The maneuver speed for the existing flap position is indicatd by the numbered flap maneuvering speed bugs."
Thats Boeings viewpoint, I would be amazed if Airbus and other manufacturers did not have a similar standpoint but I do not have the written word to quote on that.
My company SOP amends this slightly by advising that you select the next flap setting, during extension, when within 10 kts of the maneuvre speed for the existing one to avoid unecessary auto-throttle movement. Always assuming you are at a point when you want the net flap setting. We are also permitted to fly the usual ATC speeds when required by them which may be a bit above the maneuvre speed. Boeing elaborate to similar effect later in the FCTM.
I very much agree that we should not play structural engineer that is precisely why its important to know and follow the laid down guidance.
Take Flap 1 on a 737NG. The Maximum Operating Speed is 250 Kts but at Typical Weights clean speed is about 195 Kts. Thats a 55 Kt difference which at 1/2 MVsquared makes a hell of a difference to wear and tear on the flap.
During normal ops anyone trying to operate the flaps significantly outside that advice will be politely told not to with the reasons given at a suitable juncture.
So there you go Manual references, hope that helps