Forgive me, as it's several years since I flew the A320, but as you have just taken off, you haven't Gone Around, if you see what I mean, so the Take Off modes atill apply and all you have done is increased power to TO (not GA).
I am probably 100% wrong, in which case please feel free to disregard my mumblings.