If you are finding that you are so transfixed to the PFD that you don't have the capacity to flick your eyes across to scan the N1/EPR gauges and back; then perhaps you need to go-around, or improve your capacity to scan.
And I see you agree with me about the well designed thrust control of the Airbus FBW.
As you say, the Airbus system does work extremely well, and will give Vapp +/- 5kts, but if it is not doing what you need, you should take manual control - indeed Airbus specify this action if the automatics are not performing correctly.
I find the Airbus FBW auto-thrust so good that I have only ever had to do this a couple of times on older A330s, which apparently did not have the increase in auto-thrust response gain below 3,000'.
We used to be allowed to use 'phase advance'; where a little push or pull on the thrust levers would correct a "slow" auto-thrust, but presumably too many pilots did not fully understand the sequence and cocked it up and got into difficulties, so it was withdrawn. But taking manual thrust control always remains an option, in which case the thrust levers are entirely conventional and do move to match the changes of demanded thrust, (because the pilot moves them !).
"slamming" the thrust levers to go-around does indeed sound like the actions of panic, rather than the calm, considered actions of a competent pilot. The clever part of the Airbus FBW system, as you know, is that simply and instinctively clicking the levers fully forward into TOGA, switches the engines, the aircraft and the navigation systems into go-around mode, without any other switch-pressing required to select go-around.
