Don't lose sight what you brought into aviation in the first place.
Never let lack of confidence deter from your flight, keep your currency, select easier flights in that case, nicer days, less crowded airfields etc, and if all else fails, bring an instructure on board
I do share the paradox feeling though.
As a low-time pilot, I recognized that my pre-flight preparations were mostly filled with anxiety, but in some cases, I sense a little fear too, especially when I try to prepare for all the things which can go wrong.
Funny enough, I calm down when I get into the airplane, test the flight controls and the engine starts to purr. The rest is all down to what I learnt and, prepared for, emergency procedures included.
The worst situation for me are those flights, which are cancelled due to weather, aircraft glitches, etc. Then I only have the anxious part, but not the joy part.