Indeed the limitation is on the LEDs/slats and not the TE flaps. Simple case being the modified B727s with winglets with higher ceiling (I believe 41000') and farther range due to fuel savings. Part of the modification aside from chopping a few feet off the wings and slapping the winglets, is also drooping the TE flaps1-2 degrees to generate extra lift. Clearly this doesn't pose a threat at 41000'.
Some B727 crews used to do this as a little trick to fly higher by pulling the CBs for LED/slats and lowering Flaps 2 at cruise! (clearly non SOP) You might recall the accident where the unaware SO/FE, upon his return from the Lav, pusshed the CBs back in, extending the LEDs/slats putting the airplane on its back into a dive.....and the rest is history.
So, the limitation is indeed for the high lift devices.