It's not that uncommon - often associated with white knuckles and a vice-grip on the controls. In most cases it improves as general competence and confidence grows, so don't worry unduly.
A light touch on the controls will make things a lot smoother, and improve sensitivity on the trimmer(s).
Often pilot's who don't trim properly tend to over-control (or vice-versa), and it can be difficult for instructors to spot the problem.
Next time make a conscious effort to use finger and thumb only on the yolk/stick and see if it improves.