I don't fly the 320 but am wondering if there is an issue with the autothrust response at light weights here.
I once did a ferry flight in a 330 where the autothrust was cycling between full thrust and idle as it attempted to hold the cruise speed at FL410. The aircraft was very light. Descending to 370 fixed the problem.
Perhaps there is a similar issue on the 320 as it seems like the autoflight system didn't cope very well in the initial parts of the incident.