Problems at Swanwick?
 

Like many others I'm sitting waiting for a slot this morning with some vague info of communication problems in the south Uk. Anyone shed any light on the problem?

Eurocontrol cfmu portal says phone system failure

EGTT All Sectors Due to ATC equipment (Telephone failure) Unable to change from night configuration Very High delays No improvements possible RRP' s are send out to avoid EGTT where possible

Latest update

Airspace

EGTT
All Sectors
Due to ATC equipment (Telephone failure) EGTT is unable to change from night configuration, this means working with four collapsed sectors
Technicians are working to try to solve the issue and they are giving an expected time of resolution around 1400 UTC
Delays are very high
RRP' s are send out to avoid EGTT where possible. Flight level cappings, in coordination with UK FMP, are perfomed to reduce delay for flights that can fly lower

NMOC Brussels

It means everybody has slept in.

swanwick
 

At night the number of individual sectors are reduced so as to allow one controller to work combined sectors as if they were one big sector; because the traffic loading is less at night. ie one controller will work 3 daytime sectors as one large sector at night
As day arrives the traffic levels increase & the sectors have to be split so as to be capable of handling the traffic.ie the 3 sectors are now split & are worked by 3 different ATCOS.
Apparently, the problems arose when trying to switch back to the daytime system . ie systems didn't work as they should do.

Result of an insufficiently tested system upgrade possibly?

From reading the NATS release here http://www.nats.aero/news/technical-problem-update/ it looks like the comms panels aren't permitting the sectors to be split out so they have to remain in their night time configurations, which means just 5 sectors for the whole of London area control.

Plenty of EGs today lads...

It's the cleaning lady unplugging the radar again so she can plug her Hoover in! :E

There's also some information on Eurocontrol's NOP - https://www.public.cfmu.eurocontrol....pec/index.html

Eurocontrol reporting it as a telephone panel problem which also prohibits the sectors being split, and it also says London area is only FOUR sectors currently!!

That's what happens when all the eggs are put in one basket.

Nats probs@Swannick
 

From my vantage point down here at Phuket, Thailand, the UK air traffic services appears to be the laughing stock of the world judging by the number of delayed flights/passengers?

ZH875, that basket being an Air Traffic Control Centre? Not sure ATCOs can work from home yet...

But pilots will be able someday:ok:

Have they tried turning everything off, waiting a while then turn it all back on?

Nothing to do with Scottish mil task being transferred to Swanwick last night?

Sir George and ZH
 

I agree with your comments re a "Single point of Failure" - I hope that your comments are not based in actual facts: but the public reports coming from our illustrious BBC are so vague, as to suggest that the problem is as yet not fully understood.

In a crisis (of sorts) a concise and proportionately detailed explanation would come in handy.

As I herd a spokesperson for "NATS" say that they were unable to activate the usual additional ATC control positions to accommodate the normal increase in morning demand - necessitating flow control restrictions.

I wonder if Joe Public will ever know the approximate truth.

I'm with NotanERIC, morning watch up till the wee small hours watching England getting duffed by the Aussies:=

I run my own telecomms business, dealing with large, mission-critical systems, and can't quite grasp how some kind of disaster-recovery redundancy has not kicked in. Something utterly catastrophic must have happened. But then, plans for such an event would normally be in place.

Meanwhile, I notice a few departures from Manchester heading off to Newcastle, then turning right, possibly just to get away. Emirates being one example.

Does anyone know if this is a problem with their PABX (telephone system) or with some bespoke intercom system? The devices at the workstations seen on the BBC reports appear to be for an Ericsson MD110 PABX.

Agree entirely with Midland 331. If someone is stating that it is a telephone line issue, with no critical back up for an ATC system, then god help us.

Like good ole RBS, ' systemic 'systems' failures, aka, we have no cash:eek:

Just love it. Obviously with everyone grounded, the only hazard to pax will be from over congestion in the terminals. Vaccines now on site- Quote from COBRA:uhoh:

There are risks with any work on operationally critical work. The trick is to manage those risks.

The trick is .....
 

The trick is to test software changes thoroughly OFFLINE before loading changes ONLINE ! IF that is the problem ....

Feel for my colleagues in UK .... and for the SLF in their cocoons waiting for Slot times !

Hope you get it sorted soon - whatever the problem :uhoh:

AC871 from Paris to Montreal departing towards Norway rather than Ireland as I write this, I was wondering why.

So if I have it right the Atcos are unable to call the next sector and coordinate properly? Is it just the internal phone system or is it the whole lot?

I do not know the full extent of the cause, but reading both the NATS release and the Eurocontrol CFMU release, the problem looks to be that the plasma touchscreen telephone system is not configuring correctly so new positions cannot be opened to split the sectors. There is no possible redundancy/back up system for a system failure of this kind.

Una, London area control do not telephone the next sector to coordinate, it is all done electronically and the telephones used for amendments or to pass other details. The only exception to this is that not every other centre/airfield are connected to the electronic coordination system.

NAROBS
 

I really hope your are wrong here !

It's probably confidential how the workload is split & configured among the "servers" and how they are sub netted, but a detailed explanation (not a contradiction of my earlier post) should be forthcoming to allay the justified concerns of the direct customers of the services provided, and of course the flying customers too.

Will it work OK as soon as it gets dark again?

ShyTorque
 

I've had systems behave just like this too - Still wondering what happened ? - probably a Rogue Ethernet card. The bigger the problem the dafter the cause.

Or maybe improperly applied foundation?

IT Centre in Bangalore stuffing around with the nightly Batch Schedule again??:O

Sorry
 

That's me being unconsciously sexist - I meant Rogue Ethernet Card.

It is surprising how small items that have been over looked, can completely bring a computer system to its knees. Back in the late 1960,s telegraph systems were stating to be automated, by telecoms companies as well as in private ones (BA and Shell being some of the first).


The start of a message started with the code ZCZC and the ended with NNNN, a problem arose one day when someone put in a message …. (Four full stops). Full stop on a teleprinter is the uppercase of N, thus the computer considered the four full stops as the end of message code. As the code for N and full stop are the same (Space, Space, Mark, Mark, Space, if I have got it right after 40 years). So it stopped the transmission and sent an error message to the engineers, as it could not understand were the ZCZC for the next message was.


The answer was to instruct teleprinter operators to insert between each full stop, the figure shift code for upper case. Therefor the computer would not think it was seeing Four N’s on the trot and the problem was solved.


Now some of you will know why you had ZCZC and NNNN on your telegrams.

Civil Aviation computers are pretty switched on. One particular airline always asked for a wheelchar to meet the aircraft... so, one day, someone fed into the computer system "<airline> requests a wheelchair". The computer was on the ball and respeonded: "Invalid message"

Dear Mr Diamond

The answer was in the second post on this thread... which as you missed it, said the following....:ugh::ugh:

You'll be asking for METARs next :ugh:

From the Guardian, That's what you get for doing it on the cheap. No autonomous back-up system?

What happened to the 'Two Centre' strategy?
 

As a retired ATC Engineer, could I ask what happened (and when) to the two centre strategy? i.e. in the event of a catastrophic failure in Swanwick, ScOACC/Atlantic House in Prestwick could keep everything moving. :confused: