A320 stall warning is not compensated for compressibility (unlike A330/350) I.e. it comes late at high altitude, well below Vsw. You’ll also have deterrent buffet well before that.
Food for thought:
Wrt to your point of pilots entering secondary stall during recovery: a point often missed is the flying characteristic in alternate law. A release of forward stick pressure to neutral is in fact commanding a return to 1G (increase in loadfactor), which may lead to an accelerated secondary stall after initial recovery.