Lightheart,
I made the switch 5 years ago at age 36, although I had been working on my licenses and building my flight time since I was 26 (1 lesson a week on my one day off). My reasoning for giving up a good paying career and taking a 75% paycut was that it is better to regret something you have done vs. something you hadn't.
I would suggest just going for your PPL initially and see if you really love it, if you still find that you have a passion for flying after that then I would say "Go for it"