If you realy want to do it,and I mean really want to, then you should do it.
I'm 38 and never persued flying in my 20's because I always found an excuse, (had a stable job, couldn't afford the training blah blah). I'm now 38 and stuck in a job I hate, you don't want to spend the rest of your career saying if only. Go for it