The problem with contingency fuel, is that the FAA, JAA and CASA all have different methods of determining fuel requirements in addition to trip fuel. Under JAA, contingency fuel may be used after dispatch (and there are different definitions of dispatch!), and may be used in its' entirety before take-off. I, for one, would not be happy to depart on the basis that all my contingency fuel has already been used, but it is legal under JAR-OPS.
It comes down to individual command decisions, as it should always be. However, in these times of high oil prices, I would expect Flight Management to put pressure on Captains not to return to the gate for refuelling, if there was still some contingency fuel on board.
"You've earned that 4th stripe, now go and use it."