Good airmanship and sound judgment will help you understand the issue. The discussion is not to fly at VLS and flaps 1 during approach but merely close to S speed. So if ATC requires 180 kts while your S speed is at 183 kts , you won't fall off the sky because of 3 kts below S. Airbus confirms that the upper limit is Vmax and Lower limit is VLS , that does not mean you will fly at those limits on a routine basis, but only in case of emergency / abnormal.
But being close to S to satisfy an ATC requirement is a different story , again sound judgment is required , depending on weather, weights etc.., how far below S?
It is not clear cut , black or white . In gusty winds that would not be appropriate, in normal condition , few knots below S , will not hurt you.
By the way on final configuration , flaps full , how far are you from VLS?