Considering it is the FACs that determine the speed protections surely it would be more sensible to turn these off rather than the ADRs and leave yourself with only one reference for airspeed info.
I think the FACs compute the protection speeds for display, only. It is the ELAC that computes those for the protection activation. Still, FACs OFF would bring us to ALTN, which indeed would solve the problem.