A Panther Navajo I manage has a nice intercom system: I can isolate myself while everyone else chatters away, myself + copilot leaving the pax to chatter, or have everyone combined. Even better, there are two audio input jacks. One in the cockpit, and one for the pax. When I isolate pilot or crew we can have our own music. The audio input mutes for any radio or intercom sound.
If I have chatty pax I tell them that when I'm busy they can expect me to cut them out of my intercom loop.