Headcounts depending on the aircraft type (the more pax, the harder it becomes) are not fool proof and open to crew error. How many times have you done a transit stop followed by a head count and had to redo and redo and redo until the count is finally right?
I agree with Hotel Tango....the whole point of having crew members at the door is not just to meet and greet but to ensure pax are on the correct flight. As an ex purser, i would happily delay the flight to do this and write on my flight report the reason why the flight was delayed i.e boarding. I am sure no one in management would even think to question a delayed departure for what amounts to security reasons.