You are forgetting one more thing about pushing vs. Pulling.
The tail plane exerts an enourmous DOWNWARD vector of lift on the aircraft (on an A320 somewhere around 40,000 lbs of force are applied DOWN on the tail to keep the nose in a steady state) That 40,000 lbs of downward force is required to be overcome by the wings as lift in addition to the weight of the aircraft inorder to lift the whole kit and kaboodle off the ground.
When you "push over" you instantly lighten the aircraft by some percentage (rather large) of that 40,000lbs. That will have a momentary lifting effect on the aircraft before the reducing angle of attack overcomes it and the aircraft starts down hill. It is quite noticable.
There are other things guys don't notice in every day flight either. You MASH the rudder for example and the aircraft will roll the otherway first (a few degrees) before the yaw developes and the aircraft rolls in the direction of the rudder.
Cheers
Wino