Good point Captjns but i would look at this another way.
The irate passenger refused to give way to the captain and insisted that the capt wait for passengers to be boarded before boarding himself.
What he didn't know that there are many checks and preparations to be done in the cockpit before the flight and by delaying the captain, the passenger delayed those actions and hence interfered with the flight. This affects the captains's job and that of the ground crew.
I think they were well within their rights to penalise the passenger for this and should've made the point in their complaint.