ATC problems for northern europe

[Acklington]

Sky news is reporting a Frankfurt airport official has said an ATC problem stopping flights to and from northern europe

[Avman]

Apparently the system at Maastricht UAC has gone titz-up.

[UAC48]

Tact/casa Information Message - Edyy
-------------------------------------
.
Ref. : Edyy Situation
.
Reg :
.
Valid : From: 1300 Until: 1800 Utc
.
Remarks : There Has Been A Radar Failure In Edyy. Currently All
Sctrs Are Regulated At 15 Percent Of Normal Capacity. Delay Per
Flight Can Reach 200 Minutes. Operators Are Requested Not To Call
For Improvement In Edyy Regulations. Cfmu Nod Are Monitoring The
Situation. An Update Will Be Sent As Soon As The Situation Changes
.
Cfmu Ops/brussels

[fly_ebos]

We were on the backup system for about 1,5 hours.
Apparantly the track server with down due to too many vfr squawks in the lower airspace (loads of gliding was going on)

[levelD]

Interresting Fly_EBOS.
But are you entitled to disclose such specific information? :cool:

[Avman]

Keep quiet levelD, or ve may have to disclose a few secrets about you :eek:

[UAC48]

Apparantly the track server with down due to too many vfr squawks in the lower airspace (loads of gliding was going on)In this case, that's not a RADAR failure, but a SYSTEM failure...

[Lon More]

In this case, that's not a RADAR failure, but a SYSTEM failure...

but you wouldn't expect a Flow Person to know the difference :O


(although what happened was the system was overloaded with radar info)

[On the beach]

Sounds like a ginormous hole in the Swiss/Dutch/Eurocontrol cheese.

[BrATCO]

Not that big, the hole... as flow management procedures have been implemented. (Will you have another slice ?)

Can it be called a system "failure" ?
Some systems can deal with up to N tracks. They just choose a random track and delete it when they reach N+1. You lose a track, but the system still works. That's the way it was created. Not a failure...

Usually, N is big enough to deal with everyday's life.
And beancounters will tell us : "Statistics show that this event did not occur ! At least, it shouldn't occur more than once every 10000000000000(ish) years..."

Would the capacity loss (85 per cent ?:eek:) have been this huge with an ATC system based on paper strips (not as much automated) ? :E

[Lon More]

Would the capacity loss (85 per cent ?) have been this huge with an ATC system based on paper strips (not as much automated) ?

It having been a track server problem, the availability of strips would probably have been of little help. The nature of the route structure makes conflict detection using a paper based system almost impossibly complicated.
If you wanted to regress to the steam age it would be necessary to revert to an airpace route structure that last existed back in the 1980s (when it was decided that they were irrelevant to the system and structure as it developed) By the time this had been promologated and all the flight plans refiled the system would have long been up and running again (IMHO by the way; I've been out of the loop for some years now)

[andrijander]

I'm with Lon on this one. Also what would be our limitation on a daily basis if we were running on strips? An educated guess (to call it something) could well be 20% less traffic a year. We can't operate on strips anymore, it'll be way too busy managing strips i.s.o. managing airplanes.

Perhaps is time we upgrade our server. Or we implement some mitigation procedures to keep a lid on it. There's no perfect system, but it'll be criminal to know about a problem and not fix it.

I guess it'll be top priority though (back at work today, let's see what's up).

A.

[BrATCO]

When you're used to deal with steam, that's not such a big deal when the system "fails" and starts providing hot water.
My (off-topic ?) question was more about cheese layers. From my point of view, removing paper strips (in order to "improve" the system) was just putting a big number of cheese slices in a "fail-unsafe" (by definition) computer. Hence the big loss of capacity when there's one squawk too many.

[wolf_wolf]

So where are the good places to go gliding around central Europe? (southern Germany, Switzerland etc)

[PeltonLevel]

Some systems can deal with up to N tracks. They just choose a random track and delete it when they reach N+1. You lose a track, but the system still works. Seems like a scary option to me! Particularly if it does it in in a busy area.

[BrATCO]

If N is known the number of tracks can be scanned. When it's too close to N, the supervisor will be warned and he will use ATFM procedures to avoid the override. ATCOs will be advised too.
If N is reached, one can use the backup radar system (based on 20 years ago's simpler technology, N is not the same). Strips are still available, automatic co-ordinations still work (however, not with the same interface).
No real big change in procedures.

[Lon More]

So where are the good places to go gliding around central Europe? (southern Germany, Switzerland etc)

somewhere with good mobile phone coverage - ask ATC Watcher :E

[Spitoon]

Quote:
Some systems can deal with up to N tracks. They just choose a random track and delete it when they reach N+1. You lose a track, but the system still works.

Seems like a scary option to me! Particularly if it does it in in a busy area.
So how else do you think they work?

What's scary perhaps is that it sounds like the system in question fell over rather than degrading the service in a known and managed manner.

[Lon More]

IIRC the track server is one of the older bits of a system which was designed to fail progressively.
One of the problems with Conspicuity Squawks is there is no requirement for mode C info with them. This makes it very difficult to filter them out of the system, especially if the system also supports a conflict detection and alert program

[eglnyt]

I suspect it was the random part that Peltonlevel found scary as I think you'll find he has a fairly good idea of how these things work.

There will always be an N, the important thing is you know what the value of N is and you know exactly what the system will do when it reaches that point. If it drops tracks you really need to know the rules by which it will do that, just randomly dropping tracks would not be acceptable to most safety engineers or the Safety Regulator.

What we don't know from the brief details on this thread is whether the MUAC tracker actually failed or was dropping so many tracks that it was unusable.

[UAC48]

And the problem of the "N" value is (or will be more and more) the "N x N" of STCA, MTCD, labels anti conflict, ... aso...

[UAC48]

FDPS Problem on Friday 20 August 2010 at Maastricht UAC (http://www.facebook.com/topic.php?uid=94040819450&topic=14165)http://static.ak.fbcdn.net/rsrc.php/zBS5C/hash/7hwy7at6.gif


http://profile.ak.fbcdn.net/profile-ak-snc1/object2/315/88/q94040819450_64.jpg

EUROCONTROL

This is what exactly happened last week in MUAC when they suffered equipment problems:

On 20 August 2010, at 14:37 CET EUROCONTROL's Upper Area Control Centre in Maastricht, providing air navigation services to the upper airspace (above 24,500 feet) of Belgium, North-West Germany, Luxembourg and the Netherlands, suffered equipment problems.

As a result capacity was reduced and air traffic flow and capacity management measures introduced (70% reduction at 14:45, reduced to 50% at 15:23 and 30% at 15:57). At 15:20 gradual recovery started resulting in normal capacity at 16:21 (CET).

Total delay for the situation was 9367 minutes.

The Flight Data Processing System keeps track of all flights contained in its radar coverage. There is a technical limitation in a part of the system for keeping track of flights on the same SSR code. The system was tracking too many identical transponders codes, resulting in many error messages. The system reported these error messages constantly, eventually enough to block the "heartbeat" message getting to the LMCF (Local Monitoring and Control Function). The LMCF automatically monitors the health of Maastricht systems and initiated a FDPS kernel swap-over and restart. The standby kernel suffered the same fate, forcing the use of the backup system.

After identifying the reason for the crash, a modification was activated on the tracker on 25 August to mitigate the issue and to prevent last Friday's problem from occuring again. A software solution will be implemented after extensive testing on 9 September 2010, to definitively prevent this type of occurrence from re-occurring.


FDPS Problem on Friday 20 August 2010 at Maastricht UAC | Facebook (http://www.facebook.com/topic.php?uid=94040819450&topic=14165)