I'm told - and tend to believe - that you can damage aluminium ailerons by gripping them in the wrong place. If you find the rivets to indicate the position of a rib in the elevator then hold the aileron at this position, then there's no danger of squeezing it and causing metal fatigue at the trailing edge.
What I have seen is ailerons with patches over the trailing edge, where this had presumably happened.