Wasn't there a thread about the same thing except it was about Jet Airways ?
Either way, in the past I've had a gate agent tell me he was in charge while the aircraft was on the ground, I promptly packed my bags along with the other two pilots and walked off the flight.
We came to an understanding in the jetway very quickly.