you are just driving around an autopilot in normal law when hand flying a bus.
And that's exactly the law that was in use when two crews crashed airbuses during go-arounds. Exactly what makes the control surfaces move is irrelevant. Take out the AP (and more importantly the ATS) occasionally and practice your IF.