Problems at Swanwick
Beady Eye
Join Date: Feb 2001
Location: UK
Posts: 1,495
Likes: 0
Received 0 Likes
on
0 Posts
1261
All the LAN's at LACC use the IBM token ring system.
Iron City:
You're quite right, neither the first nor last time. No matter how exhaustive your testing, with a system as complex and large as LACC there are BOUND to be bugs which only surface when you drop onto the operational system and let the 'customers' lose on it.
We do our best but we are human and not perfect.
All the LAN's at LACC use the IBM token ring system.
Iron City:
You're quite right, neither the first nor last time. No matter how exhaustive your testing, with a system as complex and large as LACC there are BOUND to be bugs which only surface when you drop onto the operational system and let the 'customers' lose on it.
We do our best but we are human and not perfect.
Join Date: Apr 2001
Location: A Virtual World!
Posts: 451
Likes: 0
Received 0 Likes
on
0 Posts
Just looking at the easyJet Website, it talks about:
"Following a significant failure in the UK's air traffic control system this morning, there has been a subsequent failure in the Brussels Air Traffic Slot Co-ordination Management System. "
I presume they mean CFMU - any details please?
"Following a significant failure in the UK's air traffic control system this morning, there has been a subsequent failure in the Brussels Air Traffic Slot Co-ordination Management System. "
I presume they mean CFMU - any details please?
Join Date: May 2002
Location: uk
Posts: 4
Likes: 0
Received 0 Likes
on
0 Posts
The problem was due to 1 ATC workstation computer.
When it started up after the DDC, it had problems communicating with other computers in the system.
When it came to split out the sectors from this workstation, it couldn't copy it's Flight Data to the new workstation. So when ATC tried to use the new workstation, it received it first flight data update and goes "I don't know about this flight, I'd better restart".
Only the sectors on that sectors on the 1 problem workstation were affected, but nobody could tell what was the problem from the Monitor System. So they didn't split the other sectors.
This is a problem which has been around for a while, but has been deemed my management to be not urgent, and isn't due to be fixed for another year.
When it started up after the DDC, it had problems communicating with other computers in the system.
When it came to split out the sectors from this workstation, it couldn't copy it's Flight Data to the new workstation. So when ATC tried to use the new workstation, it received it first flight data update and goes "I don't know about this flight, I'd better restart".
Only the sectors on that sectors on the 1 problem workstation were affected, but nobody could tell what was the problem from the Monitor System. So they didn't split the other sectors.
This is a problem which has been around for a while, but has been deemed my management to be not urgent, and isn't due to be fixed for another year.
Join Date: Jun 2000
Location: EU
Posts: 413
Likes: 0
Received 0 Likes
on
0 Posts
Management made these problems; let management fix them!
I hope they don't fix it. More days like today (i.e. having time to read The Herald cover-to-cover) would be much appreciated. Thanks, Richard!
I hope they don't fix it. More days like today (i.e. having time to read The Herald cover-to-cover) would be much appreciated. Thanks, Richard!
Beady Eye
Join Date: Feb 2001
Location: UK
Posts: 1,495
Likes: 0
Received 0 Likes
on
0 Posts
1261:
Its not management who made the problem, its just a consequence of a very complex system. No chance of 'them' fixing it, thats down to the engineers and programmers at the coal face who take considerable pride in their work and take it personally when things 'fail' to work as they envisaged.
Its not management who made the problem, its just a consequence of a very complex system. No chance of 'them' fixing it, thats down to the engineers and programmers at the coal face who take considerable pride in their work and take it personally when things 'fail' to work as they envisaged.
Join Date: May 2002
Location: UK
Posts: 4
Likes: 0
Received 0 Likes
on
0 Posts
DD&C
Looks like this system problem could have been avoided totally if the day support engineers had been in - but that costs money - at least £1000 total and heh, they don't even work for Swanwick anymore because NATS management don't need 'experts' for COTS systems - and yes, the bit that broke is COTS.
TAG be warned.
TAG be warned.
Join Date: Jun 2000
Location: EU
Posts: 413
Likes: 0
Received 0 Likes
on
0 Posts
Take3Call5,
I think that Tanglefoot has hit the nail on the head. I know that it's the guys at the coalface who will always end up digging us out of whatever hole we're in, be it an engineering or ATC one!
I was referring to management metaphorically "fixing it" by allocating the resources required (both in terms of cash and manpower) to do the job properly with a new (and unproven, at least operationally) system.
The first step on the road to recovery is, after all, admitting that you're sick
I think that Tanglefoot has hit the nail on the head. I know that it's the guys at the coalface who will always end up digging us out of whatever hole we're in, be it an engineering or ATC one!
I was referring to management metaphorically "fixing it" by allocating the resources required (both in terms of cash and manpower) to do the job properly with a new (and unproven, at least operationally) system.
The first step on the road to recovery is, after all, admitting that you're sick
Last edited by 1261; 17th May 2002 at 18:25.
Join Date: Apr 2002
Location: In the South
Posts: 81
Likes: 0
Received 0 Likes
on
0 Posts
Join Date: Jan 2001
Location: I sell sea shells by the sea shore
Posts: 856
Likes: 0
Received 0 Likes
on
0 Posts
WHAO THERE....
Lets see if I've got this straight. Todays chaos was caused by a KNOWN problem that can occur on a a SINGLE workstation that our beloved management KNEW ABOUT but deemed "not urgent".
Is this TRUE?
BEX
Lets see if I've got this straight. Todays chaos was caused by a KNOWN problem that can occur on a a SINGLE workstation that our beloved management KNEW ABOUT but deemed "not urgent".
Is this TRUE?
BEX
Controversial, moi?
What frightens me is that this is the third time that I can recall in recent months where total chaos has been caused by systems failures in ATC.
The cost in money terms to all airlines but in particular to my own is astronomical.
That is before we look at the loss of goodwill from passengers.
Where will it end?
No criticism of the ATCOs caught up in it, you must be as frustrated as we pilots are.
The cost in money terms to all airlines but in particular to my own is astronomical.
That is before we look at the loss of goodwill from passengers.
Where will it end?
No criticism of the ATCOs caught up in it, you must be as frustrated as we pilots are.
Join Date: Feb 2002
Location: London
Posts: 21
Likes: 0
Received 0 Likes
on
0 Posts
Once again the system fails and the ops staff (atsa, atco and atce) are left to clear up the mess!
Funny thing though, for the first time that I have seen, the TV news said that the SWANWICK computer had a problem, where usually they blame the one at WEST DRAYTON.
Change in attitude from the management? They are usually so proud of their very expensive system in Hampshire, that I am surprised they didn't blame West Drayton again.
Or is that just the same computer?
Now I'm confused.
Funny thing though, for the first time that I have seen, the TV news said that the SWANWICK computer had a problem, where usually they blame the one at WEST DRAYTON.
Change in attitude from the management? They are usually so proud of their very expensive system in Hampshire, that I am surprised they didn't blame West Drayton again.
Or is that just the same computer?
Now I'm confused.
Join Date: Feb 2001
Location: Hants, UK
Posts: 1,064
Likes: 0
Received 0 Likes
on
0 Posts
It's all semantics to cover up for a system which seems to be teetering on the edge of stability.
Everett is correct in that this is the first failure which is attributable solely to the NERC system which has brought the place to a halt during the day. What he skates over, however, is the fact that the first NAS failure was corrected relatively quickly, but the NERC system had to remain off for the REST OF THE DAY because of the perceived difficulty in getting it back on line again. That in my book is a failure of the NERC system as it failed to do the job for which it was designed: move air traffic more efficiently than we did at LATCC. Had we still been at LATCC on that day we would have been up and running again in about an hour.
You might not also be aware that at about 1230 local today the NAS link failed for about 2-3 minutes and we all prepared to go manual. This was shortly after we had heard that the TACT computer at FMP Brussels had also failed. Delays peaked at about 178000 minutes total, I think. That's 123.6 days!!
All of this points to a system which is inherently unstable, and cannot cope with inputs which it doesn't like. The computer upon which I am typing this has protection in it which stops me doing things which will make it crash. It cost just under £1000. Why can't a multi-million pound system do the same???
Everett is correct in that this is the first failure which is attributable solely to the NERC system which has brought the place to a halt during the day. What he skates over, however, is the fact that the first NAS failure was corrected relatively quickly, but the NERC system had to remain off for the REST OF THE DAY because of the perceived difficulty in getting it back on line again. That in my book is a failure of the NERC system as it failed to do the job for which it was designed: move air traffic more efficiently than we did at LATCC. Had we still been at LATCC on that day we would have been up and running again in about an hour.
You might not also be aware that at about 1230 local today the NAS link failed for about 2-3 minutes and we all prepared to go manual. This was shortly after we had heard that the TACT computer at FMP Brussels had also failed. Delays peaked at about 178000 minutes total, I think. That's 123.6 days!!
All of this points to a system which is inherently unstable, and cannot cope with inputs which it doesn't like. The computer upon which I am typing this has protection in it which stops me doing things which will make it crash. It cost just under £1000. Why can't a multi-million pound system do the same???
Join Date: Aug 1999
Location: EGCC
Age: 74
Posts: 979
Likes: 0
Received 0 Likes
on
0 Posts
Where were the airline updates?
Firstly thanks for all the work that ATC and airline personnel do during these difficult times, it does however lead me to ask one question.
As a corporate travel agent, I have spent today fielding calls (until I finished at 2030bst) from travellers stranded at Heathrow, and other european airports, asking what is happening to their flight. When I check Flifo in the BA and BD systems there are no updates, not even to say the flight was cancelled, and I then have to start searching through the Net for airport screens (which mostly only show arrivals) to try and find out what is going on.
In this day of supposed technical automation I always feel at a loss as to why, when the data is most required, it is not available. The public service voices always seem to be saying that travellers should contact their airlines, however it is then normally impossible to get through on the phone lines, and their agents - who SLF expect to know all - are stuck as well.
Oh well, back to the drawing board and another week that will probably see us sorely tried by more distruption (only joking) somewhere in the world that will cause havoc - whether it be air, rail or any other means of transport.
Have a good weekend all and keep on smiling!
Scottie Dog
As a corporate travel agent, I have spent today fielding calls (until I finished at 2030bst) from travellers stranded at Heathrow, and other european airports, asking what is happening to their flight. When I check Flifo in the BA and BD systems there are no updates, not even to say the flight was cancelled, and I then have to start searching through the Net for airport screens (which mostly only show arrivals) to try and find out what is going on.
In this day of supposed technical automation I always feel at a loss as to why, when the data is most required, it is not available. The public service voices always seem to be saying that travellers should contact their airlines, however it is then normally impossible to get through on the phone lines, and their agents - who SLF expect to know all - are stuck as well.
Oh well, back to the drawing board and another week that will probably see us sorely tried by more distruption (only joking) somewhere in the world that will cause havoc - whether it be air, rail or any other means of transport.
Have a good weekend all and keep on smiling!
Scottie Dog
Join Date: Mar 2000
Location: Bermuda Shorts and Cessna Caravans
Posts: 242
Likes: 0
Received 0 Likes
on
0 Posts
In this day of supposed technical automation I always feel at a loss as to why, when the data is most required, it is not available.
Join Date: Dec 2001
Location: www.tiovicente.com
Age: 44
Posts: 399
Likes: 0
Received 0 Likes
on
0 Posts
******.....
And I thought that I was having a bad day!
Hope you all manage to sort this one out and just to let you know that one of the SLF appreciates what you are doing/going through.
Thanks guys - have a beer on me.
Hope you all manage to sort this one out and just to let you know that one of the SLF appreciates what you are doing/going through.
Thanks guys - have a beer on me.
Join Date: Jun 2001
Location: United Kingdom
Posts: 27
Likes: 0
Received 0 Likes
on
0 Posts
How many times can this happen again until something really nasty happens? We`re just about to enter our busiest period of the year - even after September 11th this summer will be AS busy as last year, if not busier! We need to have more confidence in the system and know that it will not let us down - we just can`t do that at present. I expect all the boys and girls did an excellent job today (like they ALWAYS do in extreme circumstances) with picking up the pieces and, knowing what they`re like, no doubt injecting a bit of humour into the situation! Well done all!!! Whose Watch`s turn is it to cop it next???
Join Date: Sep 2000
Location: Aotearoa
Age: 54
Posts: 63
Likes: 0
Received 0 Likes
on
0 Posts
Spare a thought for the unlucky Paddys that have to deal with the resultant flak every time there is a problem with the London computer. This is the third time in as many months that there has been an explosion of traffic through the Dublin and Shannon FIRs.
Mind you, its not just when the computers refuses to work at Swanwick that these problems occur. Its a daily occurance, every morning with the eastbound NAT London impose gallons of reroutes on the Irish so that they get what they want coming into English airspace.
In effect, Ireland has taken over the complexity of the airspace so that London don't have to deal with it. How much longer can NATS struggle on before the whole thing sits down?
Mind you, its not just when the computers refuses to work at Swanwick that these problems occur. Its a daily occurance, every morning with the eastbound NAT London impose gallons of reroutes on the Irish so that they get what they want coming into English airspace.
In effect, Ireland has taken over the complexity of the airspace so that London don't have to deal with it. How much longer can NATS struggle on before the whole thing sits down?