There's nowt so queer as folk!
I wouldn't be without my (very quiet) mouse clicks. The feedback they give me tells me when my mouse click has succeeded.
If the sound irritates you, maybe just turning the volume down very low would help and you wouldn't lose the benefit the clicks provide.