I think it's far more worrying that despite verbally handing control to the PNF twice, the PF continued to make inputs. The concept of a single pilot in control at any given time is a fundamental aspect of the Airbus operating procedure and it should be drilled into cadets and pilots as soon as they start their type rating.
Yes, the fact that the PNF could not see or feel that the PF had not relinquished the controls can be considered a shortcoming of the design, but ultimately the PF should have known that "you have control" means immediate hands-off-the-stick unless or until control is explicitly handed back. Even in the old Chippy I used to fly, if I was physically stronger than my instructor I could have overpowered his inputs - and had I done so and lost control as a result, few would argue that the brute-force aspect of the old cable controls was at fault.