Not sure how an airbus works but in the aircraft I fly we quite often use IAS mode to hold an airspeed. If there are gusty winds it can be quite uncomfortable as the autopilot will chase the airspeed meaning lots of pitching up and down to try and maintain the required IAS. Could be why you felt a sharp pitch down.