Because the speed holding ability of the bus is quite lacking in certain phases of flight. Being comfortable with disconnecting both or either the AP and ATHR can save a lot of headaches with flap over speeds etc.
ive personally seen it both go trending way below green dot, and also well into overspeed territory had myself, or the officer not intervened.
I.e. Airbus golden rule No 4