The other salient factor is that gliders are often in a competition, not just enjoying a gentle cross country soaring flight. The pilots are among the most driven and competitive people I've ever met. An ounce of their attention not devoted to flying the glider better than all the others is an once of attention wasted.
I have flown with some of the signaturies of the letter - they are top notch people, some with careers in professional aviation away from gliding - and they know their stuff. It was a well balanced and sensible reminder to the gliding community.