Whilst flap assemblies ARE 'stressed' for the max speeds, the fatigue life assumes they are used more often at 'normal' speeds, hence you do them no favours.
Why not 'plan ahead' and get the profile sorted out for that (expected) visual? Most operators also discourage excessive rates of descent near to MSA which is worth bearing in mind as you drop like a brick
. Often the best way is min clean, gear/flap to approach flap when cleared for the visual (with appropriate flap manoeuvre speeds) and set your company 'limit' on R of D in V/s. The reduced ground speed will help lose the height in the least track miles.