It sounds like you've answered your own question. If you enjoy it, keep going. It will be slow, but you will progress.
As an alternative, if you have more time than cash, then try a gliding club, especially if there's a cheap winch launch for the early stages. The skills you gain will pay you back when you return to powered flying.
Motorglider training is often cheaper than regular powered and just as good for learning the skills (some would say more so).