It's nothing to do with the Intercom. If your audio reception is fine with nobody else plugged in, and goes to pot as soon as someone else does, then your headset has an impedance match problem.
It's not electret etc - that's the microphone bit, and we haven't talked about that yet.
There is a "techie" fix, which is to compare your headset's impedance with other headsets, which might or might not get you any further forward.
The second fix is to take it back and tell the supplier it's faulty. That's what I'd do.
The third fix, which will work, is to send it to Richard Holder and have him convert it to ANR. He replaces the crap "pressure units" in the earpieces with some better quality ones, as well as adding ANR.