CC - sounds like you have a very wise head on your shoulders.
You don't have to give up on your dream. Get a decent job, live life a bit and do your flight training modular.
Takes a lot of the pressure away from the all or nothing integrated scenario and you get to live a little and enjoy your flying in the meantime.
You have time on your side.