Actually, Doze, the problem had to do with the snail's pace of the shuttle program and not physical condition.
Cooper in Mercury was 36 and White was 35 in the Gemini spacewalk. I knew White, as his brother was a classmate in my squad at USAFA and was the one that convinced me I could make it if I tried.
The Mercury, Gemini and Apollo guys moved ahead very quickly. From selection to first flight was three or four years at most. Shuttle was 7, 8 9 years.
My roomie was selected at age 40 and flew 9 years later! Of course, he had to stand down for two years due to the Challenger disaster.
but I digress......