The missing clue in all this may be what the flight directors were demanding.
There are just two seconds between the autopilot disconnect and the F/Ds becoming unavailable but during that time the change in measured pressure saw an indicated loss of altitude - 300 feet. The F/D bar may well have shown a fly up indication before it disappeared completely and the PF may have "blindly" followed it.
Later the F/D is restored in HDG/ALT* modes and HDG/VS modes which may have briefly confirmed to both pilots that their actions were correct.
It was only later that the PNF saw that the aircraft was climbing.
Also very telling is the autothrust disconnect, the realisation that climb thrust has been inadvertantly applied and then the subsequent reduction in thrust.
The PF didn't want to climb otherwise he would have remained in climb thrust.
By then he is really confused and doesn't really know what pitch attitude to set without F/D guidance.