I know it's hard to say what you'd do, since we're not in that position, but I find it odd that the captain departed the plane with the pax, leaving his FO and cabin crew in danger.
As the Captain, I would feel an obligation to stay with my crew until the situation was resolved.