As others have said, trimming properly is one of the key skills to accurate - and relatively workload free - flying. Lots of pilots try to finalize the trim setting before the aircraft has stabilised. Fair enough to wind in some trim to relieve the control pressure while it is still accelerating but don't confuse this with a final trim setting. The aircraft must have stabilised at a constant speed with power set before a final trim can be done.
Assuming your scan to monitor performance is OK, how tightly are you holding on to the control column? Your grip affects your ability to trim to 'hands off' - the tighter you grip the less you're able to feel whether you're still holding pressure on the control column. The goal is to hold the column like a lightly gripped pencil. Even better if you can hold your forefinger & thumb far enough apart so that only one of them can be in contact with the column at any time. Try using a grip with your 2nd, 3rd & 4th fingers tucked into your palm, leaving only your 1st finger & thumb to grip the column - then spread those digits far enough apart so that they can't both touch the column at the same time. You'll soon see if you're having to push or pull on the column to hold an attitude. Thumb to push, 1st finger to pull and avoid squeezing the column between them.
At times you will need more pressure on the column than either of those two digits can provide, so use more fingers to pull, or the heel of your thumb/palm of your hand to push. Return to a single digit pushing or pulling as soon as trim state allows. Remember not to squeeze though.