Stick with it - no question. Flying, and your enjoyment of it, is far to important to be deflected by incompetent instruction.
Rise above it, and find a good guy (there are lots and lots about).
When I was learning, back in the late 1970s, I had a mixed bunch of instructors. Some were appalling, but because some were good (one or two were VERY good), I learn't to ignore the tossers and book future lessons with the right guys.
I found there was no pattern to the 'good guys'. Some of the good, and some of the appalling, were ex-service, doing it for love. Some of each were also low-experience youngsters on their way to an airline career.
When I look at how flying has been such a joy to me over the near quarter century I've been privaleged to do it (like today, formation flying with another Chippy and a Partenavia at Liverpool John Lennon to drop a fellow pilot's loved-one's ashes), I say again - stick with it.
SSD