Because it adds extra workload, requires ambidextrous manipulation of controls and undermines the concept of the systems design in the first place. If a majority of Airbus pilots shared your view then it might be worth revisiting, but as that seems to not be the case - for all my opinion's worth - I don't think it is.