Mask and speakers on is the nastiest combination... I brief that the headset is to be put back on over the mask, and the intercom on only when something needs to be communicated. Must have lost a fair percentage of my hearing just from sim sessions with hot intercoms, masks on and the speakers on as well...
In normal ops - either use the headset or the speaker, not both together - especially when the intercom is hot.