Any decent data centre has an array of batteries that kick in as soon as one of the main power supplies( you'll normally have 3) fails and keep the machines running until the whacking great diesel generator (kept at operating temperature) takes over. It'll have fuel for the next 48 hours.
And you'll have your core systems mirrored.
This is industry standard.

You would think they would have removed the tweet that immediately precedes the two 'we can't do anything today' tweets which reads:
******
Summer isn't just about beaches - check our new High Life Special & plan your own 'oh my god' trip: (my italics)
******
I presume departure halls the world over are filled with BA pax saying those exact words at this very moment?

Murphy can bite you in the ankle any time. One company I worked with seemed to have done most things right, including two mains feeds coming into the building from two separate grids (three would have been better, but there wasn't a third available). What they didn't realise was that in one place in the street the contractors had run the two cables side by side, in a shallow conduit because of some obstruction underneath. Inevitably some fool with a backhoe began to dig a hole in just that place (he should have been on the other side of the street) and chopped both cables at once ... Thankfully the company did have not only a UPS that worked, but also a backup generator that started on demand!

It might be the "industrial standard" but util the power actually fails, you don't know if that generator will actually start or be switched in.
A good datacenter will have at least 2 separate ups's for A and B feeds to servers, and failover supplies for equipment that have only 1 actual power supply. But very few will have a second generator setup that will start if the first one don't. Then you have only minutes of ups power to find out why the generator didn't start. And complicated servers/storage and relational databases don't like sudden stops.

POWER SUPPLY FAILURE! That`s got to be the biggest moronic explanation from a bloke with a degree in engineering. Maybe he thinks UPS stands for that parcel lugging company and not for Uninterrupted Power Supply.

While the scale is certainly much bigger, this is not an isolated incident. Over the last few weeks I had repeated interactions with BA due to having to change a couple of long-haul flights several times. On more than one of these occasions I was asked to call back later, as their systems were down, sometimes for hours. Not good.

Just had a text message from a work colleague who was travelling back from PHL to EDI through LHR. He's now found some floor space in T5 which is going to be his bed for the night. Obviously hotel rooms are as rare as rocking horse poo tonight at Heathrow. He is not alone with this predicament and you have to wonder how much this is going to cost BA. Hopefully, it will hit them hard.

Well the CEO seems to have started out alright with a tech degree and then working with AA and Sabre BUT then he turned to the dark side and got an MBA and worked for 'Management Consulting' companies.

After that he ran a rinky dink start up in Spain and moved to that paragon of quality Vueling. Any one who ran an airline with that paint scheme (vomit on a dirty white background) must have something wrong with them.

In these instances the culture espoused by the CEO starts the rot, if its cut cut cut then line managers will think it is Ok or even their responsibility to pare back on things previously considered essentials , It is hard to resist when your job and bonus depend on following the party line or company culture as opposed to your experience or professional training. I have worked in more than one company where experienced chartered engineers were ridiculed for putting forward professional sound proposals because they were 'too belt and braces' or not in line with current best practice . The latter being something a 'manager' read in a magazine about vaguely similar equipment in a completely different industry

Last time I was dependent upon the (monstrous thing newly installed with much press hoorahing) generator kicking in when a power cut hit us was when sitting in Ghana Airways headquarters in Accra as Deputy Chief Exec of Something or Other ...

It did fire up, but lasted less than a minute because the local lads had drained the humungus diesel tank to set-up a highly profitable local business of their own :(

Not surprising Mr Walsh had little time to appear on national TV explaining the outage.He was far too busy on telephone being expertly advised by his IT consultants to unplug the router and wait 5 minutes. 🛩

Failure because of a "power supply problem" :

Our (small scale in comparison) IT system is 'hosted' in a failure-proof computer hosting centre. There are multiple power supplies, diesel backup generators, batteries, and the whole thing is mirrored in real time to a comparable centre in another continent with automatic failover and restart. Multiple redundancy, something an airline should be familiar with. These organisations nowadays may be Clever Stuff, but are standard, not hugely expensive, and shared by all sorts of major IT operations, banks, credit card companies ... and airlines. I remember the presentation.

I would like to enquire just where the power supply failure actually was so we can find how long the local public electricity supply organisation took to fix it, and which hosting organisation was involved. Journalists, over to you.

I did operational IT for a living, rising to CIO, before I went into academia. I got bored with being told that filesystems and databases wouldn't stand sudden stops (ACID properties, right?). I was expected to buy exotic database products from Larry Ellison, tended by smug contractors who had a million and one reasons Postgres just wouldn't do. So I made it a point of acceptance testing from development into production that the systems I was expected to run had to survive a sudden stop. Salesmen from Oracle and NetApp talk about journalling, so let's see it: we're going to flip the power at the time of our choice during your testing, and your product will survive it, and we'll do it again a few times for fun, or you can all go back to your offices and fix it. It's not the 1990s, and fsck isn't a thing any more. They'd whinge and whine that I should be doing an orderly shutdown, but I genuinely meant "I will go into the development lab and flip switches at random".

I flushed out any number of problems with this approach.

Doesn't the law state that pax are entitled to hotel rooms in these cases?

We've had a power outage on one of our test servers trash the Postgres database. But that was because the operating system truncated a bunch of files to zero bytes when it rebooted, so I doubt Oracle would have handled it any better.

That's the only system that doesn't have a battery-backed disk cache, which almost certainly would have prevented the problem.

CEO Cruz has Navitaire experience. You would think he'd know the advantage of decentralized and independent systems, but maybe he either could justify the regular but well planned and scheduled downtimes sample Ryanair had for changeovers, upgrades and maintenance. This could be just the incident to spur him on to implement changes, if he gets WW's continued trust.

Perhaps the management should read this thread for some expert advice on how to run their IT systems - it seems that the world's supply of experts' opinions are here for the taking.

Hardware is cheap, Linux is cheap - buy cheap, get cheap. Power failures, totally unacceptable, but billed as an acceptable risk against cost.

If this were Japan, (No single critical system failure in the lifetime of the product), the President would be bowing and sobbing, before being up for the chop, literally.

In the West, disgrace in business is unheard of, can't see Willy doing the honourable thing. Probably he'll take a bigger wedge this time for future protection.

Imagegear

Well he has just texted to say he is being told everybody has to leave T5, he can't stay there overnight. He has nowhere to go so he might have the pleasure of a night in a police cell 😄

Not sure what they would do if there are no hotel rooms left, it's not like they could fly them to Birmingham for the night to get a room

Planefinder shows BA215 in the air, off Heathrow circa 21:40 BST off to Boston.

Saturday 27th May 2017.