We (BA) use ANR headsets on our Airbus fleet (A319/20/21).
The headsets are non removable from the aircraft.
ANR not used during start and pushback. Intercom used for ground crew comms.
After Push and ground crew clear, Headset worn on both ears, and intercom used for all flight deck comms, for remainder of flight.
Intercomm must be latched on for t/o and landing.
ANR headsets may be removed after engines shutdown.
Not aware of any safety related incidents directly caused by ANR use, but fairly easy to miss RT calls when you first use it and not used to it. I'm pretty sure when ANR was introduced a while ago now, that the volume of aural alerts had to be increaded by engineering as those alerts don't come through the headset, but the speakers.
HTH.