I've had many horrible instructors. I think your were not particularly unlucky with that flight, you probably have been quite lucky to have good instructors in the first place.
As long as you fly for pleasure, change school if it's about learning new skills, and let the clown make his show if it's a checkout flight.
When you make it your trade to fly, an horrible instructor is another story.