Do it!
What are you considering, which direction? I left a lucrative office career for flight school at 35 years old. I flew for several years with a PPL before making the jump. After finishing flight school during COVID (2020), I worked an office job for two years before finally landing a flying job. While the job itself is less stressful, the lifestyle is demanding and hard. The barriers to entry for an airline pilot are high, but once you break through, finish the type rating, and complete LIFUS, it's worth it.
Regrets? Every career has pros and cons.