Tend to get a similar problem with the Intercom volume turned up high perhaps it's more noticable with the ANR headsets. My only suggestion is to try and turn down the I/C volume as there seems to be a point with the type I'm flying where the background noise leaps up.
It's always struck me as strange that there is no Squelch function on the intercom