From personal experience, knowing what a fighter pilot in another aircraft is doing with his controls by watching his aircraft bears no resemblance to this scenario. It is far easier to work what is going to happen by watching the controls first. There is no way the PM could have known what the PF was doing with his stick by watching the motion of the aircraft; all he would have been able to see was rapid rolling and pitching with the stall warning blaring on and off. A 200+ tonne, totally stalled airliner does not behave or react like your standard fighter being flown at normal speeds and it is naive to think that the other crew members could ascertain the PF's control inputs with any degree of confidence.
I have no doubt that if AF447 had had conventional control columns, there would have been much better SA amoungst the crew, which includes the captain when he re-appeared, about what was actually being done on the flight controls and so the possibility that one or the other would have taken over much earlier or at least increased the intensity of criticism.