Whirlybird is quite right to bring us back to the initial question. I agree that annual or biannual medicals do not adequately assess age or determine the time to stop. They merely eliminate pilots who have specific pathologies.
Having spent decades watching my colleagues age with varying effects, insight and outcome, I remain convinced that you cannot rely on the individual concerned. I believe that close colleagues almost always notice the deterioration and are aware of decreasing safety, but rarely raise concerns out of a misplaced sense of loyalty.
A number of organisations have suggested that with increasing age there is a need for a short certificate signed by a colleague confirming that the individual is not unduly impaired by age. The frequency of certification increases with age.
This is far from easy. Definition is difficult. We have to change perceptions so people feel they are helping and not letting colleagues down. In flying there is the issue of pilots flying alone, but I suspect that many of us have seen ageing pilots on the ground who obviously find things more difficult.
So perhaps the response to OP is to ask your fellow pilots if they think you are safe, invite them to fly with you from time to time, and not to be frightened to be open and ask them if they think you are past it