I had a similar problem in Miami a few months ago.
We the crew had been delayed 1 hour at a time for 3 hours for a trip from Miami to Buenos Aires.
We arrived at the gate to the same response from the passengers, The gate was locked so we had to wait awhile with this going on, After the agent arrived at the gate I took him unto the leader and told him, At this moment he was not going with us to Buenos Aires, I told him to go into the corner and sit down and not make any further nuisance of himself and that may make me change my mind.
He went into the corner and did not make a sound.
He boarded the flight with no further problem.
I was on a line check at the time; the checker said that this was very interesting to watch.
My point to the passenger was that the crew had already be up for 5 hours by the time they arrived at the airport and they could do without any additional stress from frustrated passengers, after all we could have decided not to go into discretion they would have had another 12 hours delay.