BBC1: "Breaking News - computer failure grounds UK flights"(Merged)
Join Date: Apr 2002
Location: London UK
Posts: 171
Likes: 0
Received 0 Likes
on
0 Posts
Where oh where is the "Hurin" Sector?
Join Date: Apr 2002
Location: Roxton, UK
Age: 47
Posts: 331
Likes: 0
Received 0 Likes
on
0 Posts
There's a good Matt Cartoon in the Daily Telegraph today. Whilst the 'joke' is fairly predictable the way the character is sitting and the expression is brilliant.
DeepC
DeepC
Join Date: Jan 1999
Location: London
Posts: 199
Likes: 0
Received 0 Likes
on
0 Posts
Dezi
Actually, it may well do. If the software has a problem, then the software causes the failure. Very easy to blame the software if it is at fault.
NAS was brought back on at around 0300, but the problem will not necessarily appear until the system is tested under load. That happens when the transatlantic rush comes in at ... 0500. At 0300 there was probably hardly anything in the sky.
Where do you get the 30 minutes figure from? Last time that i can remember a software build being taken out it took 45 minutes.. depends who is on duty in the computer dept (or so they tell me!) If it takes 30 minutes, you also need to include the time ittook to realise that here is a problem, the time taken to rectify the probelm and the time taken to restart the machine. On top of this, once we are in a steady state of manual operations, it may not be appropriate to go straight back to fuly computerised operations.
As i understand this particular failure, HCS was only down for approx 60 minutes, LACC for an extra 15 or so. All was back to "normal" when the early shift started at 0700 local. The delays were the knock on effect....
Biggin Koksy- good to see you are still around!
It's too easy to blame new software. It doesn't explain Thursday morning's failure.
5.03 GMT is a little late to notice problems. When was NAS first brought back online?
I understand that any software updates can be quickly removed from the system. In the worst case scenario NAS would be stopped and restarted without the updates. This would take 30 minutes maximum. 30 minutes might annoy ATC but it wouldn't cause chaos.
As i understand this particular failure, HCS was only down for approx 60 minutes, LACC for an extra 15 or so. All was back to "normal" when the early shift started at 0700 local. The delays were the knock on effect....
Biggin Koksy- good to see you are still around!
Join Date: Nov 2002
Location: UK
Posts: 7
Likes: 0
Received 0 Likes
on
0 Posts
quote
-----------------------------------------------------------------------------------
NAS was brought back on at around 0300, but the problem will not necessarily appear until the system is tested under load. That happens when the transatlantic rush comes in at ... 0500. At 0300 there was probably hardly anything in the sky.
-----------------------------------------------------------------------------------
So everything was fine until 5.03GMT?
And we can believe every word of the note on the NATS website.
-----------------------------------------------------------------------------------
NAS was brought back on at around 0300, but the problem will not necessarily appear until the system is tested under load. That happens when the transatlantic rush comes in at ... 0500. At 0300 there was probably hardly anything in the sky.
-----------------------------------------------------------------------------------
So everything was fine until 5.03GMT?
And we can believe every word of the note on the NATS website.
Join Date: Sep 2003
Location: SE England
Posts: 33
Likes: 0
Received 0 Likes
on
0 Posts
So everything was fine until 5.03GMT?
However, we didn't sit there for two hours with our thumbs up our arses doing 2/5ths of
all and ignoring a problem! 
If you have a torn hamstring but walk VERY slowly (minimal traffic at 0300) you may think you're fine. However, when you break into a sprint and your hamstring gives way, (arrival of transatlantic traffic at around 0500) you suddenly realise you have a problem you may not have been aware of.
I have no idea what you're alluding to with your comments as you obviously aren't willing to accept what really did happen. I hate to disappoint you but it really really was a fault in the software. You must have some fascinating conspiracy theory so please do share it with us all.
Please do not come onto this forum though and start suggesting that NATS and every single one of us who was working yesterday were guilty of anything less than 100% professionalism and commitment to safety and dealing with the problem as quickly as possible.
Ohcirrej
Join Date: Feb 2003
Location: This is the internet FFS.........
Posts: 2,921
Likes: 0
Received 0 Likes
on
0 Posts
ing media. At no point do they seem to make the distinct point that tha RADAR didn't fall over. This Chicken Little "The sky is falling, the sky is falling" reporting really annoys me. Yesterday a guy I work with here in the Peg asked me had I heard London Centre had fallen over. When I explained it was the HCS he shrugged and said "Oh, is that all. Media made it sound much worse eh!?"
Join Date: Apr 2004
Location: England
Posts: 75
Likes: 0
Received 0 Likes
on
0 Posts
Bravo troops!
I don't work in ATC but may I just say jolly well done to those ladies and gents who performed exceptionally well during the difficulties at West Drayton yesterday.
You guys don't get much thanks, particularly from the media, and it would be nice if you did.
Thank you for keeping our skies safe.
You guys don't get much thanks, particularly from the media, and it would be nice if you did.
Thank you for keeping our skies safe.
Join Date: Jul 2002
Location: California USA
Posts: 719
Likes: 0
Received 0 Likes
on
0 Posts
"Oh, is that all. Media made it sound much worse eh!?"
Hope you're picking up some of the local flavor by now...
Join Date: Jul 2003
Location: Cheshire, California, Geneva, and Paris
Age: 67
Posts: 867
Received 0 Likes
on
0 Posts
Interestly, I was at the other end as a passenger on the day in question. There were a lot of passengers, myself included who decided that as the flights were delayed indefinately as far as the airlines were concerned that we would cancel our flights. The airlines were not able to give refunds and I wonder if my fellow passengers and their employers might seek compensation from NATS?.
Join Date: Mar 2002
Location: Euroland
Posts: 2,814
Likes: 0
Received 0 Likes
on
0 Posts
Two questions;
1. When the New Scottish Centre is completed will it have it's own independent flight data computer or will it also rely on the one and only HCS?
2. At LATCC, when HCS was offline there was no conflict alert available to the radar controllers. Is this still the case at Swanwick? and what is the situation at Scottish and Manchester?
Regards,
DFC
1. When the New Scottish Centre is completed will it have it's own independent flight data computer or will it also rely on the one and only HCS?
2. At LATCC, when HCS was offline there was no conflict alert available to the radar controllers. Is this still the case at Swanwick? and what is the situation at Scottish and Manchester?
Regards,
DFC
DFC
1 - In terms of what we expect the system to do, the answer is yes [and no] in that order.
Yes it should have it's own integrated FDP however, until all systems are migrated to our new platform there remains a need for NAS. Page 28 of our Business Plan explains some of what I'm trying to say.
As this is a couple of years old the reality of what we need to do will be in our Service and Investment Plan (SIP). Customers should have been able to access industry consultation of the SIP. For those of an inquisitive nature pop along to ERG and ask for sight of theirs
2 - HCS RDP isn't available at ScACC for controllers (though I'm a little concerned you're heading for "sensation" territory with your unprompted reference to conflict alert !!) Lets not get silly here ....
RT
Edited for spelling !....
1 - In terms of what we expect the system to do, the answer is yes [and no] in that order.
Yes it should have it's own integrated FDP however, until all systems are migrated to our new platform there remains a need for NAS. Page 28 of our Business Plan explains some of what I'm trying to say.
As this is a couple of years old the reality of what we need to do will be in our Service and Investment Plan (SIP). Customers should have been able to access industry consultation of the SIP. For those of an inquisitive nature pop along to ERG and ask for sight of theirs
2 - HCS RDP isn't available at ScACC for controllers (though I'm a little concerned you're heading for "sensation" territory with your unprompted reference to conflict alert !!) Lets not get silly here ....
RT
Edited for spelling !....
Last edited by Roger That; 4th Jun 2004 at 23:42.
Join Date: Jul 2001
Location: Fort Worth ARTCC ZFW
Posts: 1,155
Likes: 0
Received 0 Likes
on
0 Posts
Interesting to note that your operational manager didn't have the authority to stop the test on the HOST. We have that authority over here and do use it from time to time when traffic warrants it. I have seen some night where the software folks or the maint. techs want to take the HOST down for work and the ops manager sends them all home due to weather in the area.
As to the HOST software. It is indeed ancient and what a lot of folks forget that in it's infant mode (many, many years ago on the IBM 9020) it was designed to work on 80 K of core memory. Yup, you read that right, not even a meg <G>... All done on a machine taking up about 3500 square feet of space with all of the associated equipment. (don't forget that card reader and card punch <G>) The software was written in both Basic Assembly Language as well as JOVIAL. Trying to find folks who can program in that language and know it well is getting VERY difficult indeed. Not to mention that it is getting to a point where it is difficult to use it in a modern way. It will also keep you from being able to do wonderful new things with the new display equipment that you now have.
We are embarking on a program to finally replace not only the hardware, but also the old software that is keeping us back and from doing some really wonderful things with ATC software. For those who want to keep the old HOST software, it's time to let go of that old Jovial and BAL and move forward <G>...
Good luck to all of you who had to deal with the outage. I understand the issues that go along with it. We have had them here too due to upgrades that didn't catch a certain problem. ( We have 20 different computers with different site adaptation so that makes it basicly 20 different software drops.) Make for fun when trying to test the system prior to release.
take care
Scott
As to the HOST software. It is indeed ancient and what a lot of folks forget that in it's infant mode (many, many years ago on the IBM 9020) it was designed to work on 80 K of core memory. Yup, you read that right, not even a meg <G>... All done on a machine taking up about 3500 square feet of space with all of the associated equipment. (don't forget that card reader and card punch <G>) The software was written in both Basic Assembly Language as well as JOVIAL. Trying to find folks who can program in that language and know it well is getting VERY difficult indeed. Not to mention that it is getting to a point where it is difficult to use it in a modern way. It will also keep you from being able to do wonderful new things with the new display equipment that you now have.
We are embarking on a program to finally replace not only the hardware, but also the old software that is keeping us back and from doing some really wonderful things with ATC software. For those who want to keep the old HOST software, it's time to let go of that old Jovial and BAL and move forward <G>...
Good luck to all of you who had to deal with the outage. I understand the issues that go along with it. We have had them here too due to upgrades that didn't catch a certain problem. ( We have 20 different computers with different site adaptation so that makes it basicly 20 different software drops.) Make for fun when trying to test the system prior to release.
take care
Scott
Join Date: Nov 2002
Location: UK
Posts: 7
Likes: 0
Received 0 Likes
on
0 Posts
Quote from Van Der Hum
Please do not come onto this forum though and start suggesting that NATS and every single one of us who was working yesterday were guilty of anything less than 100% professionalism and commitment to safety and dealing with the problem as quickly as possible.
Unquote.
Where did I suggest that?
All I asked was why was this start up after testing different from all the others.
NATS itself says no new software was introduced so something went wrong.
I think after Thursday's chaos everyone has a right to know exactly what happened.
Please do not come onto this forum though and start suggesting that NATS and every single one of us who was working yesterday were guilty of anything less than 100% professionalism and commitment to safety and dealing with the problem as quickly as possible.
Unquote.
Where did I suggest that?
All I asked was why was this start up after testing different from all the others.
NATS itself says no new software was introduced so something went wrong.
I think after Thursday's chaos everyone has a right to know exactly what happened.
Join Date: May 2001
Location: Spanish Riviera
Posts: 637
Likes: 0
Received 0 Likes
on
0 Posts
Dezi, do we have a big inqury every time the trains are delayed?
IMHO, something failed, a set of safety protocols were sucessfully enforced and NATS managed to get everything back on track within a relatively short amount time after suffering 100 000 minutes of delay.
I'm not normally one to festoon NATS with compliments but on this occasion, well done guys.
IMHO, something failed, a set of safety protocols were sucessfully enforced and NATS managed to get everything back on track within a relatively short amount time after suffering 100 000 minutes of delay.
I'm not normally one to festoon NATS with compliments but on this occasion, well done guys.
Join Date: Jun 2003
Location: The Netherlands
Posts: 127
Likes: 0
Received 0 Likes
on
0 Posts
I was returning from Arrecife to Manchester the day all this happened and thought the worst when i heard it on the news.However a quick call to work and i found out the real situation and relaxed.In the end we were delayed 1hr, and 10mins of that were due to 2pax!!!So someone must have been working their b@lls off!!
I'd just like to say thanks,(the worst part was in Man as it took 1hr 20 mins to get the bags off the plane-just what you need!)
I'd just like to say thanks,(the worst part was in Man as it took 1hr 20 mins to get the bags off the plane-just what you need!)
Join Date: Jan 1999
Location: London
Posts: 199
Likes: 0
Received 0 Likes
on
0 Posts
Dezi
Anyone at all is welcome to come and ask questions about ATC, especially after Thursday's "incident". However, on reading through your posts, you have asked in such a way that it would appear that you are not satisfied with the answers you are getting. The HCS was shut down, it was restarted, didn't work properly, shut down again, restarted again, worked fine.
Thats it, that is the whole story. Why are you so fascinated? I notice that you don't list an occupation in your profile.
As I said, everyone is welcome, but when a poster asks the kind of questions that you do, people start to suspect that you have a hidden agenda. Usually here it infers that a journalist is digging for a sensationalist "scoop". Please tell us why you have such an interest, and why you seem to think that there is so much more to this story...
Interesting to note that you have posted 6 times in 2 years, and the only thread you have contributed to with the exception of this one was in April of this year..
Anyone at all is welcome to come and ask questions about ATC, especially after Thursday's "incident". However, on reading through your posts, you have asked in such a way that it would appear that you are not satisfied with the answers you are getting. The HCS was shut down, it was restarted, didn't work properly, shut down again, restarted again, worked fine.
Thats it, that is the whole story. Why are you so fascinated? I notice that you don't list an occupation in your profile.
As I said, everyone is welcome, but when a poster asks the kind of questions that you do, people start to suspect that you have a hidden agenda. Usually here it infers that a journalist is digging for a sensationalist "scoop". Please tell us why you have such an interest, and why you seem to think that there is so much more to this story...
Interesting to note that you have posted 6 times in 2 years, and the only thread you have contributed to with the exception of this one was in April of this year..
Operational Engineering Staffing Levels
Is anyone concerned?
Is anyone concerned?