You are not alone - a lot of post-PPL students get this - I certainly did.
The key is currency - the number of flights you undertake in a period of time.
Once you start flying every week the nerves and confidence come down. Now I get in and turn the key and go without a thought about it, but when I was doing 5 hours a year I used to have to take a deep breath at the end of the runway
I'm close to 80 hours this year alone now ...
My aim was to make flying just like driving, where I did not need to think about nerves etc but just jump in and enjoy it. I'm finally there... (EDIT: Incase people think Im cocky with this, Im not being, This is about nerves/confidence - I still learn on every flight, I still have MUCH to learn :-) )
I now often fly in the right seat with post-ppl low-hour or low-currency pilots as P1, just having me their gives them confidence but I make it clear they are P1 and they are in charge. Somehow just being there helps.
There are mistakes and there are Mistakes. Learn to know the difference between small mistakes (Forgetting to turn the landing light off) and big mistakes (busting airspace, unlocked primer, crashes)... and then go easy on yourself...
My Driving instructor used to say that passing your driving test was the first part, then once you have the paper, you start your real learning... it was true, and the same is true of flying, a PPL is just the start of your education, now you need to learn more in your own time, and experience more...
Dont get disheartened, there are other threads in this forum that are from PPL Pilots with the same feelings...
See if your flying club has a buddy scheme, or a post-ppl confidence course - (even if its with an instructor)
Most of all - dont expect to be confident and nerveless if you only do 5 hours a year
, Also a moderate level of stress can actually improve performance (So the commercial ATPL Human Performance exam states haha)
Have fun!