Quite. The average would be dragged down by all those in the general population who are obese, smokers, stress merchants, drunks, druggies etc, and those with unfortunate genes - many with a combination of these problems.
Pilots SHOULD do better than this average. But not all will, and it is those who don't beat the average that supports the other side's argument. Unfortunately for those wishing to hang in there past 65, I believe that the counter argument will prevail in much the same way as we have ever more prescriptive SOPs and regulations to cater for the the lowest common denominator.