As already mentioned the airline will usually specify the level of service it requires in the contract with the handling agent . On the whole most contracts are based on the standard contract and then specify which sections are included/excluded.
It often includes specifics such as number of check in desks at specific times, or it may be based on maximum queue lengths at say economy, business and/or first class check in.
It may include time requirements based on arrival/departure times. Such as inbound requirements for first/last baggage delivery, cargo delivery, steps/airbridge/chocks etc. There are likely to be departure targets such as doors closed by STD minus 3 mins, 5 mins etc
The contract will usually contain penalties should the handling agent not meet the requirements. These can be per flight (such as a discount or no payment for that flight) or over a period of time such as a month, more than perhaps "X" minutes of cumulative delay or % of delays etc
It is very unusual for there to be penalties in the other direction. The more usual method is that there is an agreement that the contractural penalties will not apply if a flight is more than "X" minutes off schedule.
However, it is my view that a passenger contracts with the airline and not the handling agent. The airline is responsible for managing their contracted agents - fuellers, caterers, handling agent etc.
I'd suggest that these days airlines are mainly concerned with the financial performance of the contract and assume that the contractural penalties take care of the service level.
I have certainly in the past been involved in negotiations where there was discussion about price versus service level - there was a willingness to get a cheaper price by accepting a lower standard of service.
Also (as mentioned) I have received company legal advice when either trying to advance or defend a particular position.
That advice is usually along the lines of "there is no legal reason why you cannot use that argument/take that position, should it end up in court that position is arguable/unlikely to be sustainable".
I'm afraid this is really all only a very, very brief summary of how the relationship is between airlines and handing agents.