Yes, everyone has that feeling at one time or another, particularly at first.
I did my tailwheel conversion in a Pitts S2 with Alan Cassidy, and had much the same impression as you during the first few sorties. AC's reaction to my misgivings was...."Well, so does everyone; what makes you think you're different to everyone else?".
Hang on in there.