The clue is in the definition .... Vls = lowest selectable speed. It is designated as such by Airbus and therefore is safe.
It is another matter altogether as to what is sensible when it comes to normal operations. I am (or was) of the brigade that flying below S speed [10kts or so] was fine on a short term basis in good (non-turbulent) conditions rather than take the extra drag associated with Flap 2.