the maintainance manuals/regulations permit a certain spool up time from ground idle.
one of your engines is at or close to this limit.
this difference is due to the mec fuel metering and physical engine characteristics and is more pronounced during acceleration upto about 40% N1
the PMC does not have very good authority in the 0 to 40 range, beyond that, she works like a dream.
the N1 commanded is fixed for a given TLA for given conditions, GIVEN TIME the N1 will settle at the commanded value.
the only way to get around this is to advance the thrust levers at a slower rate so that the N1 keeps pace with the N1'command' i.e. there is no 'split' or gap between the actual N1 pointer/indication and the commandN1 pointer/index (i don't know whether you have dial or vertical tape instruments).
in any case push the toga switches only after ensuring even spool up and after 40% N1
if you find that you have pushed the thr up rapidly and the engines are spooling up unevenly - just wait, holding the thr lvr position, till both engines are even before advancing them further or pushing the toga switches.
at the expense of repeating myself - remember, pushing the lagging engine thr lvr up is not going to help because the commanded N1 is the same for both at a given position, but one engine will arrive at this value slower WITH RESPECT TO TIME.