I found it interesting that despite it being a SOP how many pilots failed to advance the throttles quickly when presented with a failure of the ASI.
It's also possible that the turbulence was so bad the pilot(s) could not do anything.
In a GA plane, in severe turbulence, you really just need to hold the horizon the right way up, more or less, and if possible maintain the pitch attitude. In the Airbus context, setting 85% N1 (or whatever) and hand-flying the specified pitch angle, is probably very hard. Especially as the speed band between stall and overspeed, at FL350, is probably only about 30kt or so wide, so you have to get it reasonably right. Even reading the instruments might have been hard.
I suspect the reason so many pilots did not revert to manual mode is that they almost never have to do so for real.