The reason why you want to keep the AP on in severe turbulence
According to Airbus..."Recent severe turbulence events have clearly illustrated that potential consequences have been minimised thanks to the appropriate use of automation by the crew, mainly in keeping Autopilot engaged instead of possible instinctive reaction, which is to take over manually.Keep Autopilot engaged. Indeed, detailed studies regarding aircraft behaviour when crossing such external perturbations has shown that the less the aircraft reacts at short term to the turbulence, the better it is. Indeed, the dynamic of such severe turbulence is so, that any additional pitch down reaction to counter the initial up draught will accentuate in most cases the pitch down effect of the down draught usually subsequent to the up draught. This will accentuate the excursion in negative load factor and so increase the risk and number of injuries. To minimise the additional effect of such pitch down order coincident to the down draught, it is recommended to the crew not to react to the turbulence by short term side stick inputs corrections and to keep Autopilot engaged."
So basically, your pitch control inputs get out of sync with the aircraft reactions making things worse.