PDA

View Full Version : EGLL problems


ALLDAYDELI
24th Aug 2005, 08:47
Whats up at Heathrow currently?
Tower asking all aircraft to shut down engines until further notice and mega delays on departures.
Is it single runway ops or is the wind really that bad currently?

Anyone know whats up?

Futher to this, I am told that Swanwick Computers have failed ...!!! again

lowfield heath
24th Aug 2005, 10:06
From CFMU

THE UK NATIONAL AIR TRAFFIC COMPUTER SYSTEM HAS SUFFERED A
FAILURE.THIS MEANS THAT UNTIL THE SYSTEM RECOVERS, AREA CONTROL
UNITS IN THE LONDON AND SCOTTISH FIR/UIRS WILL BE OPERATING AT
REDUCED CAPACITIES.
.
DEPARTURES PLANNED TO ENTER THE LONDON FIR/UIR HAVE BEEN
TEMPORARILY STOPPED.
.
FURTHER INFORMATION WILL BE TRANSMITTED BY THE CFMU FMD AS THE
SITUATION DEVELOPS.
.
ISSUED ON BEHALF OF LONDON FMP.
.
FMD BRUSSELS

angels
24th Aug 2005, 10:45
BA are going to need this like a hole in h head.....

Del Prado
24th Aug 2005, 10:55
Nats statement (http://www.nats.co.uk/news/releases/2005/2005_08_24.html)

Interruption to Air Traffic Control Service (Updated 11:00)

NATS’ Flight Data Processing (FDP) system at West Drayton, which developed a fault at 09.30 this morning was restored to full operations by 09.50. The restrictions that had been imposed on flights - and which led to delays for some travellers – have all now been lifted and our controllers are working with airports and airlines to clear the backlog and restore the schedules.

Throughout the period NATS controllers were able to operate normally using radar and radio systems which were unaffected.

NATS emphasises that safety has not been compromised and regrets any inconvenience caused.

Regrettably, we also received news that the Maastricht air traffic control centre suffered a service interruption from 10.00 UK time this morning. We have no reason to believe that the failures are connected, but this inevitably will lead to further travel delays on flights to and from some parts of Europe. We believe that Maastricht Centre is also now fully operational.

Ian Hall, NATS’ Director Operations said: “We sincerely apologise to those who have been inconvenienced this morning. We did everything we could to restore the full ATC service as quickly as possible and minimise the delays and impact on the day’s schedules.

He added: “A new FDP system sits at the heart of our £1 billion modernisation programme. It will enable us to deliver the robust service our customers expect.”

Jerricho
24th Aug 2005, 13:35
and regrets any inconvenience caused

Not to mention delays. Which destination is that?

The Truth
24th Aug 2005, 13:52
Jerricho

NATS will add up all the delays divide by another number and conclude that it only caused an average delay (NATS Attributable) of 9.64secs....

Which is fine and dandy and well en-route to Destination. Time for another pat on the back and lunch with Boss :D

The Truth, the Whole Truth and Nothing But The Truth

Jerricho
24th Aug 2005, 13:56
Makes me happy to see I'm not the only cynical person round here ;) :ok:

Hoppity
24th Aug 2005, 14:11
Ian Hall added “A new FDP system sits at the heart of our £1 billion modernisation programme. It will enable us to deliver the robust service our customers expect.”

Ask him when its due in service!! So until thats in customers aren't getting the service they should expect?

A I
24th Aug 2005, 15:08
ALLDAYDELI

Get your facts right please. The "Swanwick Computers" did not fail. There was a problem with the FDP computer at West Drayton. This serves the whole country including Scotland.

There is a programme to replace the systems in use and Ian Hall is correct in saying FDP is at the heart of the new systems. A little misleading because the earliest date for the replacement of NAS in LACC airspace is 2012. The ScACC one is earlier - about 2008.

If anyone believes that replacing the FDP system is all that matters is sadly mistaken. There will be failures of any system (Maastricht had one this morning and their system is quite new) which will cause problems. What needs to be said is that the Watches on duty all over the UK this morning did a superb job, safety was never compromised and all was back to normal as quickly as possible.

NATS staff are moving a record number of aeroplanes (up 7.5% in June and 6.9% in July). Will people please stop sniping.

A I :\

ALLDAYDELI
24th Aug 2005, 16:23
A I, Dont shoot the messenger. That old adage.
Enough said

Jerricho
24th Aug 2005, 16:42
AI, people are sniping the sector drivers.

It's the support from the systems the worker bees receive that draws the attention.

A I
24th Aug 2005, 17:16
Jerricho.

I'm not quite sure what you mean. If by sector drivers you mean the ATCO's and ATSA's on console then I wasn't aware of any sniping in their direction.

ALLDAYDELI

I wasn't shooting you! All I tried to do was put the record straight. Swanwick did not fail. One of the systems supporting the whole UK ATC operation failed for a short time, the proper fallbacks were implemented and all was well. Some of us are just a little bit fed up when ill informed people make statements which are not true. If you had simply asked the question about delays at Heathrow then that would have been fine. You implied that you knew the answer and that things were not OK at Swanwick. That is my point.

A I

Scott Voigt
24th Aug 2005, 17:19
2012 for your FDP computer replacement, WOW I thought we were slow <G>... If they don't cut anymore money or run into any more issues, we are supposed to start replacing the computers at all 20 centers starting in 2009 and will probably be finished in 2010. Interesting to see what the differences are in systems.

regards

Scott

A I
24th Aug 2005, 17:25
Hi Scott,

We are going to replace the FDP computers sooner than that!! NAS is moving to Swanwick in 2007 so the hardware will be new. NATS is in collaboration with SATCA (Spain) who will supply a new system (including a new FDP system) for Scotland which should be in operation in 2008. SATCA will also supply the system to replace Swanwick and that is scheduled for 2011/2012. We will be running NAS until then.

A I

VectorLine
24th Aug 2005, 17:58
AENA are the Spanish ANSP that NATS are working with (and DFS - the Jerries) to develop SACTA - the air traffic control system.

All three are working with INDRA - a Spanish IT company to develop a new FDP system (nicknamed ITEC).

SACTA will run whether ITEC is available or not, although I believe ITEC is supposed to be available before SACTA at Swanwick

VL

BDiONU
24th Aug 2005, 18:10
NAS is the most modern FDP system in Europe, having been recently completely rewritten. The move to Swanwick puts it on new hardware but its the same hardware, so no upgrade.

All systems will fail at some point, it is impossible to expect a computer system to never fail. You just need to have sufficiently robust fallbacks and procedures in place to cover those very rare failures.

Don't hold your breath for ITEC or any other replacement FDP system in the UK anytime soon. Replacement is not as simple as the high heid yins were led to believe and they've now had a short sharp reality check which is still reverberating around :p

BD

PPRuNe Radar
24th Aug 2005, 18:30
NAS is the most modern FDP system in Europe, having been recently completely rewritten.

If it's the most modern (and by implication, advanced) in Europe .. God help us !! :p

055166k
24th Aug 2005, 20:41
Don't worry about the management person's input, this is a rumour network after all: I mean why would the world's best system need to be shut down once a week? If NATS are supposed to be buying Off The Shelf kit why is the "super" Spanish system being completely re-jigged...perhaps a major downgrade to make it compatible with and as non user friendly as the current system?
No problem on my sector this morning, I used my paper strips and we kept the metal moving......Ah that reliable old fallback...the human element!

BDiONU
25th Aug 2005, 06:12
Hhhhmmmm, basic lack of understanding being shown as to what NAS actually is and what it does ;) Please feel free to throw rocks but I suggest that first you actually take the trouble to understand what the function of NAS is ;)

055166k
All respect to the staff who kept the traffic moving, thats always going to be how things are :D Not sure what your comments are about re-jigging the Spanish system though, are you talking about CASPIAN as opposed to any FDP system?
BTW I'm not management nor a management lackie ;)

BD

skydriller
25th Aug 2005, 07:26
NATS will add up all the delays divide by another number and conclude that it only caused an average delay (NATS Attributable) of 9.64secs....

:suspect: .... :mad:

Well the flight I was supoposed to be on yesterday from southampton to bordeaux was delayed by over 4 1/2 hours......initially due to the computer thingy affecting previous inbound then due to that delay crew were out of hours etc. etc......:{

Seven Iron
25th Aug 2005, 08:37
I couldn't believe I was reading the words that said NAS
is the most modern FDPS system in Europe, then I saw
the phrase "having been completely rewritten", "new hardware"
etc and realised it wasn't an ATCO/ATSA (who work with the
beast) and sunk into standard cynic mode. Still.

Anyone want any airspace, apply c/o NATS. Large chunks of
WORLD LEADING airspace going cheap. Take some - please.
What nice WORLD LEADERS.

World Leaders? :yuk:

Seven Iron

BEXIL160
25th Aug 2005, 11:39
Hmmm questions, questions, BDIONU:

NAS is the most modern FDP system in Europe, having been recently completely rewritten.

So, if it has been completely rewritten there will be no lines of 1960s vintage 9020D in it at all? True or false?

I also note no one has yet answered the question:
why would the world's best system need to be shut down once a week?

NOTHING has been done to sort this PROBLEM out. Why not? Why are we still stuck with the WORST DD&C option? Perhaps because the Operational Staff are still coping with the aftermath of a crap management decision years before...... I have yet to see ANY NATS manager admit they made mistakes here... I HAVE seen at least one still trying to justify his demonstrable errors.

BTW has anyone EVER seen a NATS manager being suspended and offered re-training (community service)? No, thought not.

BDiONU
25th Aug 2005, 13:02
So, if it has been completely rewritten there will be no lines of 1960s vintage 9020D in it at all? True or false?

Uuummm, its not used that code since about 1990 when the processors changed from IBM360's to IBM370's.


I also note no one has yet answered the question:why would the world's best system need to be shut down once a week?
Clearing memory leakage is one answer. Also there are changes which need to be done. If you look at NAS UK Wide and not just from a LACC Centric viewpoint there are a lot of changes which go on and that don't involve LACC but have to be made in and to NAS.

NOTHING has been done to sort this PROBLEM out. Why not? Why are we still stuck with the WORST DD&C option?
Which 'problem'? NAS has nothing to do with DD&C's, you don't do a DD&C for a NAS shutdown. DD&C's are performed the way they are for a variety of historic reasons and there is no possibility of changing the way its done now. I have to ask if its really such a 'problem'? Some people on some watches have still never been involved in one. When they occur it lasts for about 4 hours and each watch could expect to do 2 or 3 a year, only at night and only when traffic levels have been reduced.

I cannot answer on behalf of management except to say BTW has anyone EVER seen a NATS manager being suspended and offered re-training (community service)? No, thought not. Dr Rob Witty got fired very recently and other managers are being 'motivated' to avoid getting any 'personal coaching' from the Red Barron.

BD

PPRuNe Radar
25th Aug 2005, 13:44
Knowing what NAS does for us users, then I offer the following as its poor points.

1. Crap and unwieldy user interface which is not intuitive or forgiving of errors.

2. Inability to cope with any routes which are out of the ordinary. Everything has to be 'adapted' and I don't believe it has the capacity to cover every possible scenario which we see on a regular basis. Hence people spend an inordinate amount of time frigging the system and trying to get something meaningful out for controllers.

3. Extremely poor processing of direct routes, unable to provide the ATCO with anything meaningful except 7 or 8 irrelevant strips. Other FDPs can cope with airspace entry/exit which are not through recognised COPs. NAS apparently can't.

4. Defaults to 'preferred routes' between city pairs sometimes regardless of what the aircraft has actually filed. There is now a dependence placed on someone inputting the fact that the filed and processed routes differ. And if it is missed ...... ??

5. Far too many simple input errors seem to cause it to crash once a year or so. A decent system would have protections which provide a caution to the user before allowing them to make a critical error. Or simply just not allow them to do it.

6. Still unable to process PAC, REV and MAC messages thus involving staff in having to undergo co-ordination telephone calls everytime something changes after the ACT has been sent.

7. As already mentioned, extremely poor memory retention which means it needs to be shut down once a week .. is it running on a Sinclair Spectrum ZX81 or something ??

The above is a few minutes worth of thinking about, I am sure I could add many more little niggles, which I as a user have to put up with, and which I don't believe should be found in any modern and developed system, as could others.

My impression is that those who have worked tirelessy and thanklessly on the NAS 'empire' over the years now see a real threat to their domain and the realisation that their specialist knowledge and skills will soon become redundant. As such, they are extremely keen to champion NAS, some even propose developing it as part of any future system :yuk:

I strongly agree the work done now and in the past has been done as well as it possibly can by a very skilled staff group, but one day we have to wake up to the fact that the core processing philosophy which NAS has was designed in an age which did not have any of the high traffic and complexity levels which we experience today. Sure, we have continuous tinkering with the system to add on bits and pieces, but that's not the same as providing a system which has it more or less right as its basic configuration. It's inflexible, it's not easy to use, and does not provide a simple or graphic interface capability.

Seeing some of the other systems out there and what they can do in simple mouse clicks or typed instructions, I for one won't shed a tear when the NAS system is thrown out to the Science Museum ;)

Mr Chips
26th Aug 2005, 00:24
Uuummm, its not used that code since about 1990 when the processors changed from IBM360's to IBM370's.
It is quite true that the processors were changed around 1990 - which is when "9020" became "Host" or "HCS". As I recall, the name change was because the new processors (IBM4381s from memory) were "hosting" the old software.

Up until I stopped teaching FDP courses fairly recently, there were still "patches" in the system that had been there for a while.. I can even tell you who was responsible for writing them!!!

NAS IS an unwieldy system, and fully understanding it has always been a black art.

Mr Chips
So sad to not be working NAS anymore..... ;)

BEXIL160
26th Aug 2005, 10:36
BDIONU>>>>

Granted, DD&C has nothing to do with NAS. The point I was trying to make is that Swanwick is still touted as the "Most Advanced" ATC system in the world. It is not. Without the elderly NAS, Swanwick doesn't work very well at all.

NAS shutdowns?

How come Maastricht (for example) don't have to shut down once a week?

DD&C?

Remind me, just how many DD&C options were there? And why was the one were are "stuck with" chosen?

I personally have done many more than the 2-3 a year you mention. Odd that the most advanced system requires us to reduce traffic flows that cause delays for our customers, the airlines. But apparently that's ok, and not a problem.

BEX

Minesapint
26th Aug 2005, 17:48
1. Crap and unwieldy user interface which is not intuitive or forgiving of errors.

Which one? Swanwick? or any other of the dozen or so units its been stretched to manage? Have you suggested improvements? Offered suggestions?

2. Inability to cope with any routes which are out of the ordinary. Everything has to be 'adapted' and I don't believe it has the capacity to cover every possible scenario which we see on a regular basis. Hence people spend an inordinate amount of time frigging the system and trying to get something meaningful out for controllers.

We have tried to standardise routes to standardise strip outfall. Flight plan 'A' will produce strips 'B - F' at units 'G - k'. Its because our airspace is so complex. If the routes don't suit then do something about it via ops.

3. Extremely poor processing of direct routes, unable to provide the ATCO with anything meaningful except 7 or 8 irrelevant strips. Other FDPs can cope with airspace entry/exit which are not through recognised COPs. NAS apparently can't.

Agreed

4. Defaults to 'preferred routes' between city pairs sometimes regardless of what the aircraft has actually filed. There is now a dependence placed on someone inputting the fact that the filed and processed routes differ. And if it is missed ...... ??

Agreed and defined by ops reps. Again, take it up with them and we will fix it!

5. Far too many simple input errors seem to cause it to crash once a year or so. A decent system would have protections which provide a caution to the user before allowing them to make a critical error. Or simply just not allow them to do it.

Which decent system? SACTA? Eurocat?

6. Still unable to process PAC, REV and MAC messages thus involving staff in having to undergo co-ordination telephone calls everytime something changes after the ACT has been sent.

NAS is being replaced so management (and the lackies really are a pain here) will not spend money on it. PAC, MAC and REV and half done in NAS so, again, MAKE SOME NOISE!!

7. As already mentioned, extremely poor memory retention which means it needs to be shut down once a week .. is it running on a Sinclair Spectrum ZX81 or something ??

Shut down once a week due memory? I don't think so. Try for testing as 'management' will not invest in a proper test bed (actually they may now be doing this at the CTC).

My impression is that those who have worked tirelessy and thanklessly on the NAS 'empire' over the years now see a real threat to their domain and the realisation that their specialist knowledge and skills will soon become redundant. As such, they are extremely keen to champion NAS, some even propose developing it as part of any future system

Maybe so in some quarters. Lets see what the new system can do. iTEC (there isen't one) stands for interoperability through european cooperation. Does that sound like an FDP system to you. Its SACTA with tools - maybe tools like IFACTS! SACTA cannot cope with the London FIR - period!

I strongly agree the work done now and in the past has been done as well as it possibly can by a very skilled staff group, but one day we have to wake up to the fact that the core processing philosophy which NAS has was designed in an age which did not have any of the high traffic and complexity levels which we experience today. Sure, we have continuous tinkering with the system to add on bits and pieces, but that's not the same as providing a system which has it more or less right as its basic configuration. It's inflexible, it's not easy to use, and does not provide a simple or graphic interface capability.

So get involved and do something about it. Is there a thread developing here? Most ATCO's just do the job and go home.

Seeing some of the other systems out there and what they can do in simple mouse clicks or typed instructions, I for one won't shed a tear when the NAS system is thrown out to the Science Museum

Which ones? Do tell. Have you seen them actually working in a complex ATC environment like ours? No! didn't think so.

COP's out of the blue are illegal. Defined COP's are required for OLDI processing, even the IAA should know that!

Seven Iron
27th Aug 2005, 16:42
Minesapint said :
Most ATCO's just do the job and go home.

Too right and that's all I intend to do - my job. What exactly do you have in mind?

Minesapint
27th Aug 2005, 16:56
Simply that lots of people, ATCO and ATSA, are ready to critisize but very very few are actually willing to make positive contributions.

Complaining is easy, NAS IS %^$£*g complex and so will iTEC be by the time it (SACTA) has been modded to manage the London FIR, so any help is always appreciated! Most ATC staff do not understand just what FDP systems do, they use it.

Much better that complaining

Take PAC, MAC, REV. Is there a requirement for these? Not at LACC apparently and not at all for MAC and REV!!! If the IS a requirement then say so.

Euroc5175
27th Aug 2005, 19:05
Minesapint,

PAC is a requirement for TC for Manchester TMA departures once nPC comes into existence on SACTA. This has been flagged to the CASPIAN FDP team.

Minesapint
28th Aug 2005, 11:14
Aware of that. Its not for use at LACC though - no requirement apparently. The point is that although MAC and REV (and any other OLDI message) can be done in NAS there is actually no requirement for it OR management don't wish to spend the money.

DFC
30th Aug 2005, 13:42
"Why does it have to be shut down once a week?"

Surprised no one has answered that question which we asked a few years ago as operators and were told the simple and logical answer;

There is only 1. If you want to fiddle with that 1 you have to take it off line.

Other places have 2 or even 3 computers. 1 working, 1 on standby and 1 for playing with. That is why they don't have the same problem.

It is ironic that we file our FPLs and RPLs to two computers that are not just independent and backed-up but located in geographically different areas. However, that feeds to a single point of failure in London.

We were told that the plan was to have 1 new computer for all the UK in Scotland's ACC and another in London's ACC and they would provide back-up for each other. IS that still a plan or are NATs going to save some money and just by 1 system?

Regards,

DFC