I feel very under pressure when I'm with him almost as if he's waiting for me to mess up so that he can comment on it
Sounds very similar to my experience when training for my IR.
Hours of comments and criticisms and reams of notes on paper - I felt like I wasn't progressing at all.
My instructor was off sick one day and I flew with the deputy CFI - who thought I was almost ready for my Test!!!
Change instructor is probably the best initial advice.
Then see how it goes ...
Don't 'throw in the towel' just yet!