I think the reason is that the roll limit is different whenever you are on hdg or nav mode thus making that noticeable wiggle when switching between them.
They talk about it here:
A320/321 AP bank angle limits
Airbus Bank angle
Last edited by pineteam; 25th Dec 2017 at 16:10.