Single engine taxi out is very common. On our fleet of just under 100 Airbus A320 type aircraft, with a long taxi out, single engine is the norm usually.
On a light A319, the aircraft can be quite over powered whilst taxiing with two engines, meaning you need to constantly let the speed build up, then slow down to around 10kts using the brakes (as per the official Airbus way of breaking). This can be both distracting, as sometimes you feel the aircraft is trying to run away with you, and can heat up the brakes rather quickly. One way to ease the problem is to taxi out single engine.
The fuel savings are enormous and are the main reason for single engine taxi, however. On an average taxi out, you can save 80-100kg of fuel, times by 100 aircraft doing 4 to 6 sectors a day..... You do the math.
It happens far more often than you might think.