Yes, you're mad. We're all mad!
I did it fixed wing as a career-changer. Currently working as an Instructor earning next to nothing compared to what I used to earn...and I sold the house to fund it. If I get the best job offer in the world tomorrow, I still won't have the money I used to have. How am I going to fund my retirement?? Dunno!! Flying's great most of the time but flying for fun on a PPL isn't the same as flying for a living. Think carefully before you leap. It might look glamorous but it ain't really.