You can break them
I have often thought about this subject during my aviation career. Many years ago when I was a teen living in PNG I actually witnessed a fatal accident in C182 dropping parachutists at a strip not far from Rabaul in New Britain. It was over 40 years ago so my recollection may be a bit vague. It appeared that after dropping two chutes he was returning to drop two more who were on the other side of a small Cu cloud which the pilot chose to fly through. Unfortunately they, the first two parachutists, had opened early and were caught in an updraft so that when he exited the cloud the chutes were directly in front of him. As a result he took violent evasive action with elevator which snapped the tail off the aircraft. It still hit the chute with a female attached. Both she and the drop pilot perished with the instructor exiting so late that I believe he was injured because his chute only just deployed. Very sad.
The point being I guess is that if you pull hard enough on any control you are at serious risk of breaking the aircraft. As a result I have always been very gentle with control inputs during my flying career. I always take another 10 or 20 knots of all limitations even flap and gear extension because I believe it will give you that extra margin.
Groggy