I'd tell the instructor that whilst you respect his opinion, you're flying circuits in a manufacturer approved configuration for which you have been appropriately trained.
I appreciate it can be difficult to question an instructor when you're an inexperienced student, but don't be afraid to; if he's a good guy he'll respect you for it. If you're wrong, get him to explain why. If you're right, he should show some humility and admit it. If you get past that point and he's still on your case I think you can justify telling him to piss off and put his ego back in it's box.
Edit to say: I fly a 177 and usually only use flaps for <500m strips. The 177 requires flaps for take-off, so when going flapless during circuits you just pop them to 10deg as soon as you touch down, then check them before you rotate. No big problem.