If the rudder was out of trim, then you'd see consistent corrections in the traces prior to A/P disconnect - there aren't any as far as I can see.
If the yaw damper function requires an airspeed input to properly set the gain, it may have ceased functioning. This would likely put the BYDU on the line (but I think I remember information that the BYDU was not activated). In any case, the yaw damping function is not identical between Normal law and Alternate law (Yaw Alt). I know that we have discussed this a number of times in some detail, but I no longer remember the details. Getting forgetful in my old age I guess.
If the PF was correcting a constant roll to the right, why the sudden reversals of bank input to the sidestick? Surely a gradual and consistent left bank input would have been enough prior to stall - after which, as you say, ailerons become ineffective.
Dozy, remember the roll PIO theory? If he had come in gently on the controls, we probably would not be reading about this accident. Instead (probably through inexperience in this flight regime) he came in with ~ 2/3 of full left roll displacement and it took over half a minute before he got the wings settled down. In the meantime, the pitch went to unsustainable levels.