U.K. NATS Systems Failure
Join Date: Nov 2018
Location: UK
Posts: 82
Likes: 0
Received 0 Likes
on
0 Posts
You won't find me defending Executive pay but we may not be talking about the same person.
The CEO renumeration did dip when income dipped, remember most of financial year 2019/2020 wasn't affected and the critical summer period was the best ever. It has rebounded as traffic did, to my mind rather better than NATS staff in general who actually gave back their 2019/20 payrise when the pandemic hit.
The CEO renumeration did dip when income dipped, remember most of financial year 2019/2020 wasn't affected and the critical summer period was the best ever. It has rebounded as traffic did, to my mind rather better than NATS staff in general who actually gave back their 2019/20 payrise when the pandemic hit.
Join Date: Oct 2004
Location: Southern England
Posts: 485
Likes: 0
Received 0 Likes
on
0 Posts
Heading way off topic now. Not my decision to defend but those were very strange times. We now know how the future of aviation panned out but at the time pre vacs it didn't look anywhere near as positive. The furlough scheme offered a reprieve to many NATS jobs but it was quite clear that was going to run out long before income returned. NATS prioritised keeping it's validated controllers. Others were not so lucky. A large proportion of technical staff were on short term contracts through complex multi company contracts and they were let go straight away. NATS had long since outsourced many ancillary functions to organisations who weren't so good to their staff. A fair few NATS were made redundant although the terms were pretty good so they faired pretty well.
Join Date: Nov 2018
Location: UK
Posts: 82
Likes: 0
Received 0 Likes
on
0 Posts
Heading way off topic now. Not my decision to defend but those were very strange times. We now know how the future of aviation panned out but at the time pre vacs it didn't look anywhere near as positive. The furlough scheme offered a reprieve to many NATS jobs but it was quite clear that was going to run out long before income returned. NATS prioritised keeping it's validated controllers. Others were not so lucky. A large proportion of technical staff were on short term contracts through complex multi company contracts and they were let go straight away. NATS had long since outsourced many ancillary functions to organisations who weren't so good to their staff. A fair few NATS were made redundant although the terms were pretty good so they faired pretty well.
The subjects may change a little, but the theme throughout is identical, NATS is taking no blame at all for a catastrophic systems failure, that even you accept was inevitable - I suppose complexity will fool the tabloids.
Join Date: Oct 2004
Location: Southern England
Posts: 485
Likes: 0
Received 0 Likes
on
0 Posts
As I said those made redundant faired quite well but voluntary is an interesting term. NATS had served notice that it was going to unilaterally withdraw the relatively good redundancy scheme. In mid 2020 it looked quite possible that income would not recover and redundancies would be required further down the track possibly on far worse terms. Many jumped because of that. Others realised that if a lot of people jumped they would be left in an under resourced organisation that wouldn't be a nice place to be.
Join Date: Nov 2018
Location: UK
Posts: 82
Likes: 0
Received 0 Likes
on
0 Posts
As I said those made redundant faired quite well but voluntary is an interesting term. NATS had served notice that it was going to unilaterally withdraw the relatively good redundancy scheme. In mid 2020 it looked quite possible that income would not recover and redundancies would be required further down the track possibly on far worse terms. Many jumped because of that. Others realised that if a lot of people jumped they would be left in an under resourced organisation that wouldn't be a nice place to be.
Join Date: Oct 2004
Location: Southern England
Posts: 485
Likes: 0
Received 0 Likes
on
0 Posts
Every software fault is inevitable we just don't know why this one played out with the impact it did. I expect every system to fail sooner or later, I wouldn't expect the failure of a system in the Flight Data thread to take so long to fix or have that impact so I'm waiting for the report to see why it did.
When you think that they started with a system that had been running successfully for many years and that the fault occurred on a Bank Holiday weekend, and take into account all the steps they would have to work through just to isolate the faulty code, identify the cause, work out a fix and then test it, in many ways, I am surprised how quickly they did fix it.
Parliament can help in quite a lots of ways.
Firstly it can compel NATS management to tell the truth in public, which would otherwise be sadly lacking.
Secondly HMG holds a 49% share, and parliament could require that stake be exercised to improve things.
They could also commission their own report. And could come up with another number for the cost, one that might be less susceptible to manipulation by vested interests (or at least manipulated by different vested interests).
Firstly it can compel NATS management to tell the truth in public, which would otherwise be sadly lacking.
Secondly HMG holds a 49% share, and parliament could require that stake be exercised to improve things.
They could also commission their own report. And could come up with another number for the cost, one that might be less susceptible to manipulation by vested interests (or at least manipulated by different vested interests).
Or they could modify the terms of the licence to require whatever they see fit.
They might even look at a company that makes huge profits and then comes begging for public subsidy and think, you know what, maybe they should reinvest some of that profit rather than beg from the public purse. Or they could propose a tax on NATS profit to subsidise the investment if NATS won't do it themselves.
They might even look at a company that makes huge profits and then comes begging for public subsidy and think, you know what, maybe they should reinvest some of that profit rather than beg from the public purse. Or they could propose a tax on NATS profit to subsidise the investment if NATS won't do it themselves.
Join Date: Oct 2004
Location: Southern England
Posts: 485
Likes: 0
Received 0 Likes
on
0 Posts
As I've previously stated the whole World of NATS Finance remains a mystery to me but shouldn't you use the profit after tax as a meaningful measure of what might be available to invest? In fact isn't the amount paid out in dividends the only real indicator of cash lost to investment as any other "profit" is effectively retained within the business?
Join Date: Oct 2004
Location: Southern England
Posts: 485
Likes: 0
Received 0 Likes
on
0 Posts
Or they could legislate to compulsorily acquire say 10% of each of the other shareholders' shares, thus leaving HMG as a 54.1% majority shareholder. Then require the Board (or a new Board if the current Directors seem reluctant) to immediately and comprehensively address the underlying issues.
It actually probably doesn't need to do that. It holds more shares than anybody else and the 5% in the Employee Sharetrust is effectively out of play so it already has the power to control the company by asking its appointed directors to intervene but again that would send shockwaves through the markets. Still we currently have a Government that takes decisions based on "focus groups" regardless of the effect on the economy so who knows.
Last edited by eglnyt; 5th Sep 2023 at 09:54. Reason: changed a word for clarity
Join Date: Nov 2018
Location: UK
Posts: 82
Likes: 0
Received 0 Likes
on
0 Posts
I'm not sure that's a valid way forward. There are rules about acquiring shareholdings in public companies. If you introduce legislation that steamrollers through those rules, even for a limited scope, you are setting a dangerous precedent. For a country totally reliant on other people's investment any hint that it might go all "Socialist" would be very problematic.
It actually probably doesn't need to do that. It holds more shares than anybody else and the 5% in the Employee Sharetrust is effectively out of play so it already has the power to control the company by asking its appointed directors to intervene but again that would send shockwaves through the markets. Still we currently have a Government that takes decisions based on "focus groups" regardless of the effect on the economy so who knows.
It actually probably doesn't need to do that. It holds more shares than anybody else and the 5% in the Employee Sharetrust is effectively out of play so it already has the power to control the company by asking its appointed directors to intervene but again that would send shockwaves through the markets. Still we currently have a Government that takes decisions based on "focus groups" regardless of the effect on the economy so who knows.
Anyone who properly knows the organisation knows that this isn't true (at least at a management level, not to take away from the great work the ATCOs and engineers are doing generally). The PPP - which strangely seems to be morphing into a PLC - doesn't help in gaining transparency and accountability. The main aim seems to be to keep the status quo, and the monopoly going as long as possible, or at least 'until I retire'...
The sector is about to undergo massive change with a 10x increase in aircraft (mostly uncrewed) in the next decade, and the wide scale digitisation of services. This sits poorly with a (largely) analogue organisation.
(At the risk of another 250 messages!) no one should still be running 1960s technology, especially not in a mission critical environment, and especially if you haven't provisioned for any fall backs (ignore the 'expense, complexity' etc deflections). And despite all of the posturing about 'airspace modernisation' it's the organisation that hasn't done it for sixty years that is claiming it's the only organisation that can do it now. Really, there are other approved providers of these services out there? We could go on, but there's plainly a significant management malaise in NATS - there has been for a long time - and just 'spinning the story' won't resolve the core issues.
Maybe privatisation would...?
Ecce Homo! Loquitur...
Thread Starter
https://publicapps.caa.co.uk/modalap...etail&id=12321
CAP2582: NERL Major Incident Preliminary Report
https://publicapps.caa.co.uk/docs/33...y%20Report.pdf
CAP2582: NERL Major Incident Preliminary Report
https://publicapps.caa.co.uk/docs/33...y%20Report.pdf
Join Date: Sep 2006
Location: Out of a bag
Posts: 658
Likes: 0
Received 0 Likes
on
0 Posts
https://publicapps.caa.co.uk/modalap...etail&id=12321
CAP2582: NERL Major Incident Preliminary Report
https://publicapps.caa.co.uk/docs/33...y%20Report.pdf
CAP2582: NERL Major Incident Preliminary Report
https://publicapps.caa.co.uk/docs/33...y%20Report.pdf
A small but important part of NATS’ overall air traffic control technical infrastructure, it was found to have encountered an extremely rare set of circumstances presented by a flight plan that included two identically named, but separate waypoint markers outside of UK airspace.
This led to a ‘critical exception’ whereby both the primary system and its backup entered a fail-safe mode. The report details how, in these circumstances, the system could not reject the flight plan without a clear understanding of what possible impact it may have had. Nor could it be allowed through and risk presenting air traffic controllers with incorrect safety critical information. On the day, the time taken to recover was driven by the need to identify the problem and the specific data, isolate and remove it in a controlled way, and then test it to ensure it could be returned safely into operation.
This scenario had never been encountered before, with the system having previously processed more than 15 million flight plans over the 5 years it has been in service. Steps have been taken to ensure the incident cannot be repeated.
https://www.bbc.co.uk/news/business-66723586
Martin Rolfe, chief executive of Nats, said that the system did "what it was designed to do, i.e. fail safely when it receives data that it can't process".
He described it as "a one in 15 million flight plan that we received", meaning the engineers took a few hours to work out a situation they were not familiar with.
Martin Rolfe, chief executive of Nats, said that the system did "what it was designed to do, i.e. fail safely when it receives data that it can't process".
He described it as "a one in 15 million flight plan that we received", meaning the engineers took a few hours to work out a situation they were not familiar with.
Join Date: Oct 2004
Location: Southern England
Posts: 485
Likes: 0
Received 0 Likes
on
0 Posts
So it wasn't the ageing Flight Data Processing system. It was relatively new, well compared to the systems around it, so lack of investment wasn't the issue.
it's a system used all over the place. There are no COTS products in ATC but this one is as close as it gets.
It is the sacrificial lamb in the NATS architecture, it processes and sanitises data before it gets to the more critical systems. It does have main and fallback running the same software.
it's a system used all over the place. There are no COTS products in ATC but this one is as close as it gets.
It is the sacrificial lamb in the NATS architecture, it processes and sanitises data before it gets to the more critical systems. It does have main and fallback running the same software.
Good ‘ole NAS - tried and tested technology. Not your fault.
Join Date: Oct 2004
Location: Southern England
Posts: 485
Likes: 0
Received 0 Likes
on
0 Posts
If it failed in the bit that does the conversion then technically you can still blame NAS because of the need to have that function. But it appears to have failed whilst parsing the route which I suspect may be one of their more standard software modules.as lots of the variants will need to do that. That said IFPS which is a relative must do some of that in order to know when to send the plan to each state but it didn't fail.
Last edited by eglnyt; 6th Sep 2023 at 07:53. Reason: typo
Join Date: Nov 2008
Location: Scotland
Posts: 16
Likes: 0
Received 0 Likes
on
0 Posts
Frequentis Comsoft from 2018 - based on earlier good experience
Join Date: Jul 2013
Location: UK
Posts: 22
Likes: 0
Received 0 Likes
on
0 Posts
The report, submitted earlier this week to the Civil Aviation Authority, identified an issue with a flight plan processing sub-system called Flight Plan Reception Suite Automated – Replacement (FPRSA-R).
A small but important part of NATS’ overall air traffic control technical infrastructure, it was found to have encountered an extremely rare set of circumstances presented by a flight plan that included two identically named, but separate waypoint markers outside of UK airspace.
This led to a ‘critical exception’ whereby both the primary system and its backup entered a fail-safe mode. The report details how, in these circumstances, the system could not reject the flight plan without a clear understanding of what possible impact it may have had. Nor could it be allowed through and risk presenting air traffic controllers with incorrect safety critical information. On the day, the time taken to recover was driven by the need to identify the problem and the specific data, isolate and remove it in a controlled way, and then test it to ensure it could be returned safely into operation.
This scenario had never been encountered before, with the system having previously processed more than 15 million flight plans over the 5 years it has been in service. Steps have been taken to ensure the incident cannot be repeated.
A small but important part of NATS’ overall air traffic control technical infrastructure, it was found to have encountered an extremely rare set of circumstances presented by a flight plan that included two identically named, but separate waypoint markers outside of UK airspace.
This led to a ‘critical exception’ whereby both the primary system and its backup entered a fail-safe mode. The report details how, in these circumstances, the system could not reject the flight plan without a clear understanding of what possible impact it may have had. Nor could it be allowed through and risk presenting air traffic controllers with incorrect safety critical information. On the day, the time taken to recover was driven by the need to identify the problem and the specific data, isolate and remove it in a controlled way, and then test it to ensure it could be returned safely into operation.
This scenario had never been encountered before, with the system having previously processed more than 15 million flight plans over the 5 years it has been in service. Steps have been taken to ensure the incident cannot be repeated.
https://aviation.stackexchange.com/q...supposed-to-be
correctly
What can possibly go wrong? It's not like it is an unknown issue.
, the system could not reject the flight plan without a clear understanding of what possible impact it may have had