Not so. Many people get concerned that the aircraft has not moved away from the stand (or is still boarding) at the scheduled time for departure. Of course it is nice to arrive on time, but how much nicer it is to arrive a little earlier.
If you drive from say Manchester to London and anticipate a three hour drive arriving at 15:00, it is great to be able to leave a few minutes early and to arrive after only two and three quarter hours.
Same with air travel. I would love the aircraft to actually leave the stand on time, as it means my overall door to door travel time is reduced, irrespective of whether the plane actually lands on time. Why sit on an aircraft not going anywhere.