It might sound harsh mate but, if you're dreading the next lesson when you're only at the stage of PPL and ATPL theory, then you need to know it's going to get a lot worse before it gets better. The IR requires laser like focus and it's much more brutally unforgiving than anything that's gone before, specially when you're spending multiple hundred quid per hour to sit in the aircraft, you put a hell of a lot more pressure on yourself.
Now I don't know you from Adam and I don't know if you're actually a good guy suffering from a confidence issue, or if you're actually really one of the people who won't make it due to basic shortcomings in ability. What I can say is that in my experience, with 90% of the people I taught, it was a mind game. Nobody "knows" if they're good enough to make it until they've actually made it. I had my doubts, and god knows I'm far far from a skilled natural pilot, my personal dread thing was throwing the aircraft round doing stalls / spins / aerobatics, they scared the !!!! out of me but I persevered and ultimately made the grade.
However, you've got to really really really really want it, and basically you've got to be the most pig headed or the most desperate, and be the last one to quit in the face of adversity. Who gets the job is ultimately a game of last man standing as much as it is who's got the natural ability.
I dunno what the answer is. Start by going and buying a pint for one or two of the instructors you respect - really respect - and airing your doubts to them, and ask them for some brutally honest feedback, that might help your decision making process