Carry an extra ton of fuel will burn around 30-40 kgs of additional fuel per hour depending on the engines fitted and performance deterioration factor. The fuel burn on the A320 is about 40kgs per minute, less when light at higher altitude and more when heavy low down.
Roughly a minute’s flying per ton carried which isn’t much when compared with ATC delays but when multiplied by flights per day, over a year you’re talking about a fair number of hours.