Got my JAA CPL/FI at the tender age of 46, FAA CPL/multi/IR the year before. Doing my JAA IR and IRI this year, am now 49. I wanted to fly from the age of 9 but allowed various diversions to be too effective.
Bottom line- if you want to do it enough you will, if not the flying world will get on fine without you! Who's loss?
PF