I suspect the Captain thourght that the bags had been loaded. The weight diference would have showed up on his sector fuel burn thats about all.
This happens now and then when the ramp staff lose the plot.(send 2 & 4 pence we are going to a dance)
Last one I heard of was when an a/c got surrounded by police. The crew were then told that the guns (4 tons of) had not been unloaded at the previouse base.
For those of you not crew this happens as the holds are often the last doors to be closed. The flight crew only go on what they have been told by the dispatchers. Personally I would be very surprised if any Captain would go without all the passengers Bags. Just not moral is it!