HH,
A wise instructor told me once that "any pilot that has never felt scared should not be flying"
Flying, at speed, way above the ground is the teritory of birds not men, it is so far beyond the natural enviroment of man that a normally functioning human brain is bound to feel some kind of fear. If you have never felt fear (to some extent) whilst flying you probably have a brain disorder!
It is the fear of concequences that makes us do checklists, it is fear that makes us keep a close eye on the ASI on short final, it is fear that reminds us to apply carb heat regularly.
Once these behaviors are entrenched and are automatic responses, the fear that we feel disapears but if you forget, for example, to keep a close watch on the ASI on final you can bet your life that it will be fear that reminds you to look!
In short fear is what keeps us safe.
Sounds to me like your instructor thinks you fly just fine,
You seem to have good airmanship in spades, IE going around when you felt things wern't right instead of trying to land from a less than ideal approach, and landing and taxiing back when you felt that you were unhappy to go on. These are the actions of a safe pilot, not one who is going to "lose control of the situation/aircraft"
I think its a case of "back on the horse" for you, with your instructor for a couple of circuits but then back on your own.
Next time will be a breeze, honest.