It's doable. ....but you will need absolute committment to the journey and, as others have said, the financail return is less than optimal in the beginning.
My first flight was a TIF at age 41, followed by self funded, part time training up to nearly CPL. I then handed the finances over to an extremely understanding wife and finished the CPL on a full time basis. Straight off to Instructor school and got a job with the school where I did all my training. Filled in the Instrument rating in following years and added training endorsements as I could afford them.
27 years later I have just over 10,000 hours with about 6,000 hours instructional and 4,000 hours charter, aeromed and night freight. Didn't make a fortune but wound up comfortable enough and, more importantly, scratched the itch to see if I could do it. As a good friend once said to me, " Never die wondering".
Good luck and enjoy the journey!