Why is it scary? If on an approach the pilot decides to do manual thrust and forgets to add thrust, the speed comes through Vls and hits alpha prot, regardless of whether ATHR is engaged or not the Alpha Floor protection will kick in and give TOGA thrust. If the pilot then disengages the ATHR again then the protection is again removed. It makes good sense.
Not entirely true. If the T/H is armed, and speed is deselected, then the bar on the speed button will illuminate and the thrust levers will advance to maintain selected speed.
If the A/T is disarmed, with insufficient thrust to maintain the approach path, then the jet will stall.