and expect my colleague to have his hands there as well to verify
The last thing you want is two pairs of hands doing something to the thrust levers. A recipe for confusion on who the hell is flying the thing. In any case the co-pilot has a set of eyes. He just needs to look at the thrust levers to know what is going on. Riding the controls as some captains do if the co-pilot is PF and vice versa, is most annoying, distracting and unnecessary.