Having done that about 7 years ago, I would say start with a PPL. Figure out if flying is the thing that captivates you, or if it's the act of observing flight that does it for you. (No shame if it's the second one!)
If after you've got your PPL you're still excited to go flying, then work on your CPL - buying a plane at that point may not be a terrible idea - and start researching what you want to do next. Instructing? Charter? BizJets? Corporate? Airline? Every one of those - other than getting to fly a plane - is basically a different career path. See above as to what happens when you pick poorly (and, unlike with many jobs, once you're in one, the opportunity cost of job hunting is exceedingly high - except maybe instructing, as it seems to be the pipeline into many jobs now)
I was 27 when I left a well-paying, but soul-crushing, job. Through doing a road trip to get my first job, to finding out what a "**** operator" is first-hand, to getting laid off and doing odd jobs for a year while doing yet another cross-country road trip, to getting a 'round-the-clock medevac job, to moving to Australia for love and marriage, to doing odd jobs while converting my license for a year, to job hunting for a job here (just to realize I might have to start at the bottom again)... I've had lots of ups and downs in aviation, but one thing I haven't had was regret of the choice to get into it.
That said, everyone is different. If the above sounds bad to you (and, to be honest,
it's not what I thought aviation was going to be), and you decide not to get into it... There's no shame in that either. You've got a career you enjoy, and a hobby you love. That's not a bad place to be!
Hopefully, that makes your decision a bit harder!
PS: While I'm not claiming everyone's experience will be as
trying as mine... I'm also not claiming to know what your luck is like. Clearly, mine is lousy, so I try not to rely on it!