Go Back  PPRuNe Forums > Misc. Forums > Passengers & SLF (Self Loading Freight)
Reload this Page >

BA delays at LHR - Computer issue

Passengers & SLF (Self Loading Freight) If you are regularly a passenger on any airline then why not post your questions here?

BA delays at LHR - Computer issue

Old 29th May 2017, 06:26
  #221 (permalink)  
 
Join Date: Jul 2009
Location: korat thailand
Age: 80
Posts: 136
did I miss it??

No mention of Willie Walsh. Left Aer lingus in a Huff. Lots of strikes and lock-outs at BA whilst there, and now this.
crippen is offline  
Old 29th May 2017, 06:31
  #222 (permalink)  
 
Join Date: Feb 2001
Location: The Winchester
Posts: 5,942
Originally Posted by southern duel View Post
Yes it did affect other carriers but my point is that as the "home carrier" they should have had some sort of contingency in place. Instead of running around like headless chickens and not fulfilling their duties to the public.
I'd agree it would be nice to have a sure fire contingency plan to keep the whole operation running on schedule no matter what the problem...but how do you keep the whole operation running without a glitch if say there is e.g. another Buncefield?

You can never guarantee to keep the operation going, it's how you handle things when the music stops that then counts...and where I would agree with you was that on Saturday one had the impression there was of a lack of visible leadership and a breakdown in comms...

Last edited by wiggy; 29th May 2017 at 06:41.
wiggy is offline  
Old 29th May 2017, 06:43
  #223 (permalink)  
 
Join Date: May 2008
Location: Paris
Age: 57
Posts: 101
Originally Posted by Cazalet33 View Post
The power supply excuse doesn't quite sound credible for such a wide ranging series of failures.
They only way I can see it happening is as follows :

You're running x number of data centers each geographically remote of each and each mirroring the active Datacenter and each designed to perform a seamless horse swap.

On the active data centre DC1 you have kicked a batch job consisting of a volume of updates within the database subsystem. The power cut hits with uncommitted updates still in full flow.

DC2 takes over, continues, but because DC1 didn't have time to communicate a synch point, and as a result of network latency, DC2 resumes at the incorrect moment of processing. Now may have a corrupt database. Worse, subsequent batch jobs in the queue may use it as input, spreading the damage. Even worse, this corruption has been faithfully reproduced on the other data centers, so you now have no hot swap capability.
Nialler is offline  
Old 29th May 2017, 06:48
  #224 (permalink)  
 
Join Date: Jan 2008
Location: It used to be an island...
Posts: 233
Originally Posted by BEagle View Post
So waiting for ba to sort out claims through fistfuls of receipts won't be an adequate response. The CAA has clearly decided that ba's passenger delays were not caused by 'extraordinary circumstances'; the EU law on flight compensation uses the term 'extraordinary circumstances' to refer to situations where delays or cancellations have been caused by things that are not the responsibility of the airline, whereas the airline clearly is responsible for its IT systems having sufficient resilience and redundancy.

Thus full compensation under EU law must be paid.
I'm afraid I don't agree with the CAA saying that. The CAA is saying that passengers are "passengers affected ... are protected under EU law", and this law provides for both a duty of care (in all circumstances) and compensation (in some circumstances). So the EU law will protect all passengers in case of travel disruption but may only require a duty of care.

The CAA is being reassuring to passengers and warning BA (and other airlines who may be watching) not to neglect the duty of care, rather than making any comments on compensation.

The duty of care does include getting passengers on their way as quickly as reasonable possible, for example - not excluding booking onto other carriers, which is often a sticking point for tight-fisted airlines. This may be one of the things the CAA is trying to remind the airlines of, again without any reference to compensation.
nicolai is offline  
Old 29th May 2017, 07:00
  #225 (permalink)  
 
Join Date: Feb 2003
Location: BHX LXR ASW
Posts: 2,205
Although the rot really set in when Bob Ayling took over
Correct. Also wasn't he responsible for the demise of the BA franchises that were working so well for BA? To me that's when the rot really set in. Since then BMI, GB Airways, Maersk to name a few have all gone.
crewmeal is offline  
Old 29th May 2017, 07:31
  #226 (permalink)  
 
Join Date: Apr 2006
Location: ESSL
Age: 76
Posts: 60
BA will have to pay

Originally Posted by nicolai View Post
I'm afraid I don't agree with the CAA saying that. The CAA is saying that passengers are "passengers affected ... are protected under EU law", and this law provides for both a duty of care (in all circumstances) and compensation (in some circumstances). So the EU law will protect all passengers in case of travel disruption but may only require a duty of care.

The CAA is being reassuring to passengers and warning BA (and other airlines who may be watching) not to neglect the duty of care, rather than making any comments on compensation.

The duty of care does include getting passengers on their way as quickly as reasonable possible, for example - not excluding booking onto other carriers, which is often a sticking point for tight-fisted airlines. This may be one of the things the CAA is trying to remind the airlines of, again without any reference to compensation.
In a roundabout way what the CAP is saying is that the airline scam of using a blanket extraordinary circumstances to avoid pay compensation is over the ECJ has ruled that the practice of unforseen technical delay being used to exclude claims when the aircraft was late from maintenance is no excuse. Neither will an act of God unless Willy Wonka is still considered to be .God
FlightCosting is offline  
Old 29th May 2017, 07:49
  #227 (permalink)  
 
Join Date: Oct 2009
Location: Australia
Posts: 357
Originally Posted by Sonny_Jim View Post
Would not be surprised in the slightest if the 'power supply problem' was actually a botched upgrade.'.
The "Hi Tech" parks in India where outsourcers run their operations from OFTEN have power outages (once a day is common). Believe it or not, they also have rats running across the floors.
p.j.m is offline  
Old 29th May 2017, 07:50
  #228 (permalink)  
 
Join Date: Oct 2000
Location: London, UK
Posts: 75
According to someone who knows someone ... this is not a simple power failure. There has been a "failure" of core routers via the power supplies/backup power supplies to these routers. A targeted attack has not been ruled out. In some areas the network has up to six levels of resiliency, and these outer resilient layers have been targeted first.

This has been across multiple data centres, not all in the UK. In fact the primary UK data centre was the last to fall off the network. The underlying applications are mostly available, but inaccessible due to the network being down.
DuncanF is offline  
Old 29th May 2017, 07:59
  #229 (permalink)  
 
Join Date: May 1999
Location: Quite near 'An aerodrome somewhere in England'
Posts: 25,818
Indeed, FlightCosting! I read the CAA statement as a firm reminder to Cruz that 'extraordinary circumstances' cannot be considered to apply in this case - he brought it on ba with his policies.

Doubly unfortunate for Cruz that amongst the passengers stuck in his Heathrow chaos there happened to be a Daily Mail journalist and his family: British Airways doesn't deserve name, says Robert Hardman | Daily Mail Online

I don't imagine that this comment from Cruz (again from the Daily Mail) will endear him to many:

Mr Cruz's email asking his workers not to comment

Guys,

either you are part of the team working to fix this or you aren’t. We are not in the mode of ‘debriefing on what happened’ but rather ‘let’s fix this mode’.

I have been answering some emails from colleagues (thank you for the support) and I just finished a video for media and another for all of us. I suppose you will have seen our Twitter account and BA.com.

Now, your interest on today’s events is well noted. I am wondering if you would like to help out? Either LHR or LGW or anyone of our stations could use you, now. If you indeed can, drop me a message and I will connect you or go straight to the airport and make yourself available. I am sure they will appreciate your involvement.

In the meantime, if you do not want to get involved or cannot get involved, I would kindly ask you to refrain from live commentary, unless it is a message of support to the thousands of colleagues that love BA as much as you do.

Thanks
Full article at British Airways' cost-cutting boss gags his staff | Daily Mail Online

BEagle is offline  
Old 29th May 2017, 08:00
  #230 (permalink)  
 
Join Date: Oct 2009
Location: Australia
Posts: 357
Originally Posted by DuncanF View Post
The underlying applications are mostly available, but inaccessible due to the network being down.
So its not a HP SAN fault? lol
p.j.m is offline  
Old 29th May 2017, 08:15
  #231 (permalink)  
 
Join Date: May 2008
Location: Paris
Age: 57
Posts: 101
Originally Posted by BEagle View Post
Indeed, FlightCosting! I read the CAA statement as a firm reminder to Cruz that 'extraordinary circumstances' cannot be considered to apply in this case - he brought it on ba with his policies.

Doubly unfortunate for Cruz that amongst the passengers stuck in his Heathrow chaos there happened to be a Daily Mail journalist and his family: British Airways doesn't deserve name, says Robert Hardman | Daily Mail Online

I don't imagine that this comment from Cruz (again from the Daily Mail) will endear him to many:



Full article at British Airways' cost-cutting boss gags his staff | Daily Mail Online

Asking staff to remain schtum in these events is standard industry practice. It's generally done for legal reasons. If someone names a piece of kit as the culprit and the subsequent enquiry reveals, for example, that it failed because it was being asked to operate outside its documented operational limits, then you will be in receipt of letters from Messrs Sue, Grabbit, and Runne pronto.


As is being indicated, there may be huge sums of money at the core of this.


Remember in this, that it seems to have happened at a time of very heavy use.
Nialler is offline  
Old 29th May 2017, 08:28
  #232 (permalink)  
Thread Starter
 
Join Date: Apr 2010
Location: London
Posts: 7,072
Looks like they are cherry picking shorthaul flights today - one each way to Aberdeen cancelled mid- morning for example

BBC this morning

British Airways passengers are facing a third day of disruption at Heathrow as the airline deals with the impact of a worldwide computer system crash.
BA says it aims to operate a full long-haul schedule and a "high proportion" of short-haul services after the outage caused by a power failure.

So far on Monday, 13 short-haul flights at Heathrow have been cancelled.

All flights operated from Gatwick on Sunday but more than a third of services from Heathrow - mostly to short-haul destinations - were cancelled.

In a statement, BA said its IT systems were moving "closer to full operational capacity", and chief executive Alex Cruz has posted videos on Twitter apologising for what he called a "horrible time for passengers". But no-one from the airline has been made available to explain the cause of the system crash, and it has not explained why there was no back-up system in place.
BA added: "At Heathrow, we have operated virtually all our scheduled long-haul flights, though the knock-on effects of Saturday's disruption resulted in a reduced short-haul programme.
"We apologise again to customers for the frustration and inconvenience they are experiencing and thank them for their continued patience."

One of the affected passengers is Ian Sanderson who is stuck in-transit in London after setting off from his home in Luxembourg to travel to Iceland.
He said he was "incandescent with rage" after being unable to rebook his flight, or speak to a member of staff.
Image copyright Twitter Image caption Thousands of customers have taken to Twitter to vent their frustration Speaking on Sunday evening, he said: "I've bombarded them with about 100 tweets in the last 24 hours. I know that's annoying but there's nothing else I can do.
"We've tried to call them on the numbers they give and all we've got is the same recorded message which then cuts off at the end."

Former Virgin Airlines spokesman Paul Charles said: "What seems remarkable is there was no back-up system kicking in within a few minutes system failing.
"Businesses of this type need systems backing up all the time, and this is what passengers expect."

BA is liable to reimburse thousands of passengers for refreshments and hotel expenses, and travel industry commentators have suggested the cost to the company - part of Europe's largest airline group IAG - could run in to tens of millions of pounds.Customers displaced by flight cancellations can claim up to £200 a day for a room (based on two people sharing), £50 for transport between the hotel and airport, and £25 a day per adult for meals and refreshments.
On Saturday, travellers spent the night sleeping on terminal floors at Heathrow on yoga mats provided by BA.
The disruption continued into Sunday, with queues building up as passengers tried to rebook flights. Conference rooms at the airport were opened to provide somewhere more comfortable for passengers to rest.

BA said Heathrow was still expected to be congested on Monday and urged travellers not to go to the airport unless they had a confirmed booking for a flight that was operating.
It said passengers could get a full refund or rebook to travel up to the end of November but recommend they use its website.
Heathrow Harry is offline  
Old 29th May 2017, 08:54
  #233 (permalink)  
 
Join Date: May 2017
Location: London
Posts: 1
Originally Posted by DuncanF View Post
According to someone who knows someone ... this is not a simple power failure. There has been a "failure" of core routers via the power supplies/backup power supplies to these routers. A targeted attack has not been ruled out. In some areas the network has up to six levels of resiliency, and these outer resilient layers have been targeted first.

This has been across multiple data centres, not all in the UK. In fact the primary UK data centre was the last to fall off the network. The underlying applications are mostly available, but inaccessible due to the network being down.
Not entirely true - the datacentres are both at LHR, not worldwide. However I think "power failure" is a fig-leaf of an explanation.
tinyGiraffe is offline  
Old 29th May 2017, 09:17
  #234 (permalink)  
 
Join Date: Dec 2011
Location: London
Posts: 7
Anyone used to work there?

I'm the transport corr at the BBC and I'm looking to talk to anyone who used to work in BA IT...I'm on [email protected]
RichardBeeb is offline  
Old 29th May 2017, 09:28
  #235 (permalink)  
 
Join Date: May 2011
Location: Girona
Posts: 230
EU261/ 2004 inter alia

Originally Posted by Heathrow Harry View Post



BA is liable to reimburse thousands of passengers for refreshments and hotel expenses, and travel industry commentators have suggested the cost to the company - part of Europe's largest airline group IAG - could run in to tens of millions of pounds.

i) Customers displaced by flight cancellations can claim up to £200 a day for a room (based on two people sharing), £50 for transport between the hotel and airport, and £25 a day per adult for meals and refreshments.


ii) On Saturday, travellers spent the night sleeping on terminal floors at Heathrow on yoga mats provided by BA.
i) These figures do not exist anywhere in the EU261/ 2004 legislation.

ii) These people cannot claim for the costs of overnight accommodation under
the above provisions since patently they did not contract said accommodation.

Harsh evaluation ? Yes indeed. But who ever believed the politicians and the HR bods when they said repeatedly that "life was fair" ?



(Indeed the simple fact that the vast majority of passengers at both LHR and LGW could not contract such provisions due to the elementary logic which dictates that all hotels will have been full within, let's say 4 hours of the incident also substantially limits the de facto bill under EU261, I would say. Though obviously the world wide ramifications must be borne in mind as well. Likewise the issue of a much smaller number of potentially much more costly compensation claims furth of EU261.)

The question of some ex gratia offer being made in the next few hours to assuage the current tsunami of negative publicity is also a factor I believe though as, unlike several posters here it would appear, I only know of the major players involved through often ignorant and biassed press publicity filtered through expensive PR companies, I am not in a position to speculate on the probability.
BigFrank is offline  
Old 29th May 2017, 09:31
  #236 (permalink)  
 
Join Date: Oct 2002
Location: London UK
Posts: 6,626
Here's what I get on the BA website, 48 hours after the "failure", when I click on Flight Information to find out how my forthcoming flight is going.
Attached Images
File Type: jpg
BA message.JPG (73.0 KB, 148 views)
WHBM is offline  
Old 29th May 2017, 09:31
  #237 (permalink)  
 
Join Date: Feb 2008
Location: London
Posts: 14
"One day's delay may lead to a thousand years of regret".


Two days delay may lead to everlasting dishonour. Three days delay . . . do not bother cookie boy, shut-up shop, go-home and practice Ludo.
NAROBS is offline  
Old 29th May 2017, 09:34
  #238 (permalink)  
 
Join Date: Mar 2014
Location: UK
Age: 71
Posts: 54
Originally Posted by Nialler View Post
DC2 takes over, continues, but because DC1 didn't have time to communicate a synch point, and as a result of network latency, DC2 resumes at the incorrect moment of processing. Now may have a corrupt database. Worse, subsequent batch jobs in the queue may use it as input, spreading the damage. Even worse, this corruption has been faithfully reproduced on the other data centers, so you now have no hot swap capability.
Easy, you mirror the uncommitted database queue as much as is feasible.
One changeover you have a procedure to immediately process that queue, and if a few bits get missed then you deal with it in the field and accept that loss.
If you have to restart procedures for certain events (passenger boarding for instance) ... then these are isolated and expedited. Meaning 5 plane loads of passengers are disembarked and reembarked.
In other words every part of the procedure is planned for.
Nothing is out of process.
And practiced, a failure is just another day at work.
If airlines do not do this, they are completely irresponsible.
This is normal.
rideforever is offline  
Old 29th May 2017, 09:46
  #239 (permalink)  
 
Join Date: Sep 2002
Location: Middlesesx
Posts: 2,071
No doubt it has been said but when the dust settles Mixed Fleet will be looking to take further industrial action as there is no resolution to the previous issues.

AC presently rallying the troops and no doubt applauding them, when it gets back to normal his senior managers will be looking to outsource and reduce more personnel.

If the longest crew trips are 7 days then clearly crewing issues may not be resolved entirely until next week. Time to start looking for an inducement to bring customers back ; drinks and catering on shorthaul?
HZ123 is offline  
Old 29th May 2017, 10:03
  #240 (permalink)  
 
Join Date: Aug 2007
Location: West London, UK
Posts: 12
[QUOTE=Caribbean Boy;9785587]Isn't BA's main data centre in Boudicea House near Hatton Cross, with an unmanned data centre in nearby Cranebank? If there was a failure of hardware or software in Boudicea House, I would have expected the operation to run from Cranebank. There is UPS in both buildings, so how could a power supply issue cause such massive failure.[/QUOTEnk.

The high-level network diagram used to be stuck to the wall over my desk back in the day. RTZ ended up being hosted by Amadeus, with resilient links to both Amadeus data centres from both BOHO and Cranebank. I don't know the power supply side of it, but I'd imagine from its location that BOHO has grid supplies from LHR's Iver and Staines feeders as well as local emergency generation.
xs-baggage is offline  

Thread Tools
Search this Thread

Contact Us - Archive - Advertising - Cookie Policy - Privacy Statement - Terms of Service - Do Not Sell My Personal Information -

Copyright © 2021 MH Sub I, LLC dba Internet Brands. All rights reserved. Use of this site indicates your consent to the Terms of Use.