But my (rather limited) experience seems to suggest that the aircraft will want to pitch up and climb whenever we hit a gust.
How would you know the direction of the gust? Surely gusts must by definition come from every direction?
Anyway:
The CoG ahead of the CoP is the first thing to think about. In this sense the aircraft is like a dart. There is nothing else to worry about in the first instance. Get some darts and throw then backwards, sideways, everyway you like. Watch the flight.
Don't worry about the direction of the gusts or anything else.
The aircraft is flying along in a certain direction and a perturbation in the air changes its attitude. It is STILL moving through the air in the same direction. In the case of pitch and yaw, the 'dart' stability restores the aircraft so that it regains its orientation pointing in the direction of travel. Roll stability is different.
Now there is a lot more to aircraft stability as understood by test pilots but I think that this is the place to start and indeed may be sufficient for a beginning pilot. As far as I recall it was for me and I have the advantage of never having progressed beyond being a beginning pilot:-)
I did not know for instance that the tailplane produced downward thrust in normal flight or indeed that a canard produces upward thrust and is therefore more efficient (uses less fuel).