Mad Bear
You are absolutely right to think the way you do and brave too.
It is too early to worry yet, but if you are still not happy in another 5/6 hours, perhaps you could consider always flying with an instructor and not gaining your PPL.
That way, you could still enjoy flying.
Whichever way it works out, good luck.