Chris. I have no direct knowledge of the effect you describe on the A320. However, I would contend that with the rudder mounted above the fuselage and the engines a good distance below, it is unlikely that the airflow from the engines would impinge on the rudder/vertical stabiliser surfaces.
I think what you may experience is the engine exhaust impinging on the (much larger in surface area) side portions of the fuselage. Also you speak of a rolling take off. Hence I would suggest that a speed increase, coupled with the jetwash acting on the fuselage could account for the effect you describe. Especially if you are referring to an increased effectivenes of the rudder in the up-wind direction, requiring a reduction in applied rudder pressure as you apply T/O power.
It's also possible, just throwing ideas up in the air here, that the upwind engine gets cleaner air than the downwind engine, allowing it to produce marginally more power and hence acting to correct the tendency to weathercock. This could easily manifest itself as an apparent increase in rudder effectiveness due to jetwash on the tail fin.
Can anyone say categorically that on the A320 the jet exhaust doesn't cause increased rudder effectiveness? Well, I'd say that dinner is pretty safe Chris, it's not easy to prove a negative. However I think I've come up with a couple of viable alternative answers.