Vikki
I started flying lessons at 17 , before I had a driving licence, and like you had to space the flying out to match money and of course weather. I also had to stop for a few years until circumstances allowed me to restart.
I never regretted starting, and I am eternally grateful that I was able to get back into the air again. The early flying experience still has happy memories.
Of course its another matter if you intended to make it a career, but there are professional pilots who have started out in their 30's and have a successful career.
As MLS said, take pleasure in each flight you manage to make.