I'd probably flown into slightly warmer or cooler air, or into a slight up-draft or down-draft, and now it was out of trim. I had to keep re-trimming.
Actually this is one of the situations where, arguably, you should NOT re-trim.
Trim, in a light aircraft, essentially determines the speed. The power setting determines whether you go up or down, or stay level, with that speed. That's simply the way the balance of forces works out.
When you enter an up- or downdraft the balance of forces is still the same so you're technically still "in trim", but you're just climbing with the air, or descending with it. If you want to prevent that, adjust power a bit and use the stick to help the aircraft settle a bit earlier in its new attitude, or use the stick to force the aircraft to climb or descend without changing power and accept a speed variation.
Retrimming in this situation should not be necessary, assuming the up- and downdrafts are temporary. Because once you exit the up- or downdrafts you'd otherwise have to trim again for a stable cruise.
And that's the whole idea about trim: you trim for the stable situation. The stable climb, the stable cruise or the stable descent. Anything temporarily is handled by accepting some stick forces without retrimming. You also don't trim while making a turn, do you?
Having said that, chasing the correct altitude, attitude and airspeed in turbulence can be really tiring and uncomfortable. If you're in controlled airspace and have an altitude assigned, or near controlled airspace and worried about descending through the base, or something like that, you have no choice. But if you're out there in the blue wild yonder it is far less stressful, and more comfortable for your passengers, just to accept the occasional altitude excursion as long as it's not too severe. In the end the up- and downdrafts will probably cancel each other out anyway.