As with any activity where some physical skill has to be integrated with some mental agility there can be problems related to aptitude.
However age alone is NOT necessarily one of those problems.
There are plenty on here who have taken up flying later in life.
As others have said above, the issue to to relax, not castigate yourself for every slight error. Above all, don't get so caught up in a set of "why on earth did I do that, why didn't I do this", questions whilst you are flying. That lessons learnt stuff should properly come on the ground, after flight, when you run through in your mind what happened and why - the the good things even more than the bad. An instructor might help you with this, with a through debrief after flight.
There is a similarity there with a musical performance - the biggest cause of performance breakdown is precisely what I've mentioned above - wondering why this or that note was wrong - which takes attention away from reading ahead and anticipating. Flying, in my experience, is quite like that - sometimes referred to as 'staying ahead of the aeroplane'. Anticipation of what will happen next, what you have to do next, is important, it takes time to learn, but it enables you to free your mind to cope with the unexpected if it comes. Practice on the ground, as others have mentioned, can enable you to inculcate this habit of thinking ahead.
Didn't mean to ramble on quite so much - but the message is, don't be despondent.