Hello,
You are in take off mode FMGS wise. Setting TOGA in this phase will not trigger the go-around mode of the fmgs. Imagine, you need toga during your t/o or initial climb for whatever reason, and the thing switching to go-around modes...("GO TRK ..."). Not nice engineers thought. Critical phase, pilot asks thrust, give him that and keep the other modes.
Setting toga during the t/o phase will just give you MAN TOGA/ A THR blue, other mode/logics/activation will remain.
As soon as another vertical mode engages (by the FMGS or by yourself), then the FMGS switches to CLB phase. I.e you push to level off for OEI cleanup : the system goes into CLB phase ! (remember target speed auto jumps to clb speed even OEI).
Then it's another story and you will get GO-AROUND mode activation of FMGS if conditions described above are met... only if you activate TOGA AFTER the clb phase has activated.
Very unusual case, less critical phase, balance between too complex software / + required response in thrust will still be achieved.
Same thing one engine out.
The system is built taking into consideration the possible need of TOGA during the FMGS t/o phase hence not activating "surprising modes".
But, as in any other aircraft, if you start playing "the more clever pilot" and I am saying this without arrogance, we all try that from time to time... (like setting toga during OEI accel level) then you might get surprises !
Enjoy,
Flyer146