PPRuNe Forums

PPRuNe Forums (https://www.pprune.org/)
-   ATC Issues (https://www.pprune.org/atc-issues-18/)
-   -   Problems at Swanwick? (https://www.pprune.org/atc-issues/529366-problems-swanwick.html)

The Many Tentacles 7th Dec 2013 14:59

What are you on about for starters?

Secondly, the airspace can't just be handed over to another centre as there's no controllers there that would be valid to work the airspace.

Thirdly, West Drayton doesn't exist anymore so it would be somewhat difficult for them to do it from there

eastern wiseguy 7th Dec 2013 15:18

Narobs

I would seriously doubt a "without notice exercise".....that is NOT how things happen in a live environment. I can see something like that as part of a TRUCE type scenario.

As TMT implies......
the remainder of your post is nonsense.

Mantovani 7th Dec 2013 15:27

I think you'll find all ATC systems have back-ups. It is only when the main and standby systems fail that there is trouble.

That is why engineering delays are so exceptional.

angels 7th Dec 2013 16:52

Folks, I don't know much about ATC, but I know a lot about management.

A while ago, someone, somewhere, decided to take a decision which would save money. Money was saved and he was rewarded handsomely. Downstream folk who pondered the wisdom of the money saving were ignored.

Now it's all gone Tango Uniform. But don't worry about our chap. He's retired on a mega-pension.

You know it makes sense. :suspect:

Self Loading Freight 7th Dec 2013 16:59

I think the VCCS is the Northrup Grumman Park Air Systems GAREX 220 - here's a press release, unfortunately with no year date, which promotes the deal and talks of ongoing relationships.

A bit more digging produced a PDF from an ICAO briefing with some high level technical details of the NATS VCCS network configuration - still marketing flavoured, but more than I've seen in public elsewhere.

Ganzic 7th Dec 2013 17:03

LHR
 
Straight in approaches all day into EGLL, bet the crews love that, but not what awaits for them at the terminal though...

Tankertrashnav 7th Dec 2013 17:04


It means everybody has slept in.

Will it work OK as soon as it gets dark again?

It's the cleaning lady unplugging the radar again so she can plug her Hoover in! http://images.ibsrv.net/ibsrv/res/sr...ilies/evil.gif
etc etc

This is my first visit to ATC issues and I'm delighted to see you're allowed to have a sense of humour on here. I tried a little joke on R & N once and their po-faced mods had it deleted ASAP and I was told to save that stuff for Jet Blast!

goldfrog 7th Dec 2013 17:13


Originally Posted by Self Loading Freight (Post 8193730)
I think the VCCS is the Northrup Grumman Park Air Systems GAREX 220 - here's a press release, unfortunately with no year date, which promotes the deal and talks of ongoing relationships.

The Park Air is the TC Voice Switch The AC switch is a Frequentis!

eglnyt 7th Dec 2013 17:24


How much would it cost would to have some reserve channels on satellite/microwave link feeding realtime data direct from the UK instrumentation to a handover control ?
Well if we assume that one centre cost £625 Million and assume we'll want a different one so that it doesn't fail for the same reason then roughly £625 Million plus the cost of some rather expensive satellite bandwidth.

ZOOKER 7th Dec 2013 17:30

NAROBS
I am certain that there is no "redundant back up next door to the main facility".

angels, I believe I have met, and had 'a conversation' with the individual of whom you speak. :E

Flapping_Madly 7th Dec 2013 17:45

Are we there yet ?

LostThePicture 7th Dec 2013 17:52


A while ago, someone, somewhere, decided to take a decision which would save money. Money was saved and he was rewarded handsomely. Downstream folk who pondered the wisdom of the money saving were ignored.
For one who claims to know little about ATC, you certainly seem to have a fair idea about current events within the operation....

Gonzo 7th Dec 2013 17:58

NAROBS,

Where are you getting your fully qualified and current ATCOs to man your mirror of Swanwick, ready to take over via satellite at a moment's notice?

highflyer40 7th Dec 2013 18:05

the thing here is this isn't a safety critical area. so there probably was no need for redundancy. all this is is a failure to be able to switch from night mode to day mode. all it means is they can handle much less traffic. sure it will cost them some money in compensation, but less than if they built in redundancy into every single item!

Ganzic 7th Dec 2013 18:06

ATC backup
 
I have been on a day visit to NATS and when I suggested what happens if their nice shiny glass roof comes down, I met a few grim faces and was told: "well we are all dispensable", and there is a backup almost next door... one can only guess what "next door" means... in any case they are not down, they have limited capacity, i suppose that a backup facility would have limited capacity as well, due to staff not normally working these sectors and possible same or lower coverage or both.

Ganzic 7th Dec 2013 18:08

Gonzo,
"Where are you getting your fully qualified and current ATCOs to man your mirror of Swanwick, ready to take over via satellite at a moment's notice?"
exactly my point

Del Prado 7th Dec 2013 18:13


Gonzo NAROBS,

Where are you getting your fully qualified and current ATCOs to man your mirror of Swanwick, ready to take over via satellite at a moment's notice?
Just use the ones that are sitting about twiddling their thumbs today. You could easily fly them....oh, wait.......

:E

Mantovani 7th Dec 2013 18:40

NATS's t w i t t e r feed says the problem has now been resolved.

Warped Factor 7th Dec 2013 18:44

The one sure thing you can guarantee on days like today is that the internet will be filled with armchair quarterbacks.

looneykeycode 7th Dec 2013 18:44

utter cobblers being talked about the swanwick problem today
 
Having read the posts for the day I can advise that its pretty much all cobblers and mis informed guesswork.
Nats have a major contingency provision for a Swanwick failure but its never going to be invoked for a problem of this scale.
The systems are fully redundant in thread terms but if the software causes a common mode problem it will affect all the redundancy in the affected systems
NATS would need to fully duplicate all systems at the controller position and the servers and networks on completely different operating systems, platform and application software to negate any common mode faults.
That scale of redundant provision with all the development, support etc overheads would be a massive additional cost and is just a complete non starter

Self Loading Freight 7th Dec 2013 18:55

Apparently the problem was "it's very big and complicated with over a MEEELION LINES of CODE!". No, I'm not kidding, that is the official reason.

Is this the message NATS wants to deliver - running something right on the edge of manageability that's so complex it can barely be understood? Blaming code size is a very strong indication of design, implementation, testing and/or management flaws - most of which will come back to roost in NATS' own eves.

I don't know the details of the system, nor would I want to speculate about stuff on which I'm that ignorant (would like to know what happened, though)... but I do know something about disaster PR in the tech world...

BARKINGMAD 7th Dec 2013 18:56

What Goes Up Might Come Down.
 
As David Gunson's epic and memorable monologue has been quoted, maybe time our lords and masters with the power to not spend the money listened to it and took heed?!?!

Apparently the BBC have been bitten by a hi-tech phone system, all mod cons, which failed and had to be scrapped at great cost to the licencepayers of UK.

Is this the ATC equivalent of FBW versus steelcables and tactile feedback.

OK l've got my tin hat on, start the incoming.....

Angels in post #49 says it better than most, and over here in austerity Blighty we're sick to death of being preached to and :mad: 'd on by bankers and politicians and so-called managers who pose as our superiors whilst failing miserably in their responsibilities and sloping off rapido to enjoy their self negotiated pensions and share options before the excreta hits the impellor.

And just WHO was the Capt Speaking caught on national UK broacast media APOLOGISING FOR THE DELAY???? I was always advised not to apologise for problems NOT of the company's making, otherwise the SLF will take the apology as admitting some sort of liability/responsibility for the delay/distress etc.

Answers on a postcard please, to my UK kennel.......

Looks like the terminal staff were failing to keep the SLF info'd which is guaranteed to hack them off big time! What has gone wrong with our ability to keep the pax in the loop, a practice which has always worked for me and others at the coalface?

hammerthrower 7th Dec 2013 18:58

The redundancy will be in place, however by moving to another site wouldreduce the capacity even more. Ops Director said on BBC News earlier 88% offlights went out... The back up operations will only be designed to clear theskies in the event of a major incident. It’s more than just internal comms too,as stated on an earlier post they need to contact their colleagues from otherANSP’s when handing over aircraft…

rmcfarlane 7th Dec 2013 19:03


Nats have a major contingency provision for a Swanwick failure but its never going to be invoked for a problem of this scale.
It's possible their contingency is build for different scenarios, but based on the info we were given - it was the failure of a single system. Until I'm told otherwise.

I'm sure there is redundancy built in the telephone infrastructure, but I'm still surprised that this brought NATS down in such a way.

Swanwick has a major training facility, virtually next door to the main site that can be switched to take control of the live environment. If that was deemed appropriate, I'm sure they would have done so.

I worked on the business continuity plan for a major call centre. During the process I was staggered to find that the UK is littered with fully functioning, but vacant call centres. Desks, PC's the lot... People throw a lot of money at this.

I'm not comparing a call centre to NATS with their bespoke kit. But the principles are the same. Diverse routing, multiple pathways, redundancy... all built in from the start.

HEATHROW DIRECTOR 7th Dec 2013 19:13

<<Swanwick has a major training facility, virtually next door to the main site that can be switched to take control of the live environment. If that was deemed appropriate, I'm sure they would have done so.>>

But where do you get the staff to man it in an emergency? When I moved from Heathrow to West Drayton with the Heathrow Approach sector, staffing was arranged so that both units were manned until after the move had been proved successful. When something breaks down you can't simply magic-up a whole ATC watch!

DaveReidUK 7th Dec 2013 19:16


sure it will cost them some money in compensation
Including, no doubt, a substantial payoff for the person whose head is going to roll for this.

Lon More 7th Dec 2013 19:22

Just when you think you have developed a fool-proof system along will come a better fool.
When MUAC was opened we had duplicated, even triplicated phone lines. Unfortunately all came together just outside the building and as there was a bit too much cable a couple of Esses were laid. Unfortunately not copied onto the drawings and forgotten about until some ground works had to be done. It went very quiet in the Ops. Room.


Interesting that the Beeb decided to talk to a representative of Ryanair in the light of some of MOL's previous diatribes against the system. Anything can be fixed if you throw enough money at it, but would Ryanair be prepared to pay the price.

NAROBS, as has been explained the problem lay in the VCCS, If you don't have the backup in the Ops. Room what do you expect staff to do; run down the road to the nearest phone box?


Swanwick has a major training facility, virtually next door to the main site that can be switched to take control of the live environment
Although I haven't been to Swanwick since it opened I doubt if this is the case. This would leave the whole system open to abuse, In any case the VCCS would be the same.

Warped Factor 7th Dec 2013 19:23


I'm sure there is redundancy built in the telephone infrastructure, but I'm still surprised that this brought NATS down in such a way.
Yes, handling around 90% of a normal Saturday's traffic to this point in time despite the problem, the company is truly on its knees :ugh:


As of 19.30 we had handled 3,250 flights today - around 90% of the traffic handled last Saturday (3,613 flights)

eglnyt 7th Dec 2013 19:45


I worked on the business continuity plan for a major call centre. During the process I was staggered to find that the UK is littered with fully functioning, but vacant call centres. Desks, PC's the lot... People throw a lot of money at this.
The problem is that somewhere between the customer and the call centre there needs to be a switch which switches the customers call to whichever of your many redundant call centres is being used. It doesn't matter how many call centres you have if that switch is the bit that goes wrong.

goldfrog 7th Dec 2013 19:55


Originally Posted by dikastes (Post 8194029)
[Nats have a major contingency provision for a Swanwick failure but its never going to be invoked for a problem of this scale]

looneykeycode
Clearly you don't know what you are talking about either!!

I believe that he knows exactly what he was talking about. The contingency system (I helped design/install it) is a 'clear the skies' system not designed for as failure like today's. The amount of traffic it could move is far lower than that which was moved in today's failure mode and the risk of swinging all the data/comms over to the contingency would require a much more dire failure before it would be even considered.

radeng 7th Dec 2013 20:27

Not being a professional in ATC matters, I'm still of the opinion that Angels in post 48 had it right. In so many cases in, so many industries, is this the case. Followed by "lessons will be learnt" and those responsible going away with a big pay off.

MYvol 7th Dec 2013 20:42

ZCZC NNNN
 
And of course there was a Greek gentleman with zczc in his name.

Capetonian 7th Dec 2013 20:49

You're showing your age. I remember the telexes with NNNN and ZCZC and the QK QD QF codes etc on the old AIRIMP systems.

Back to topic, I had friends going back to SA tonight on three separate flights, and despite dire predictions, all left as scheduled.

Self Loading Freight 7th Dec 2013 21:28

I suppose that there won't be any details of what happened made public, such as whether it was a result of the Scottish MIL integration, and that's so mired in security and politics that it would be otiose to pursue further - at least here!

Some assurance that it has been properly understood and that it won't happen again would be welcome, though. As a sleazy tech hack journo with an unhealthy interest in aviation, I'm going to have to keep digging...

expediteoff 7th Dec 2013 22:37

2 December, 2013

NATS PAYS £15.5M INTERIM DIVIDEND TO SHAREHOLDERS

NATS, the UK's leading provider of air navigation services, paid a second interim dividend of £15.5m on 2 December 2013 in respect of the year ending 31 March 2014. This brings the total dividends paid for this financial year to £62 million.

NATS (En Route) plc, the company's economically regulated provider of en route air traffic services, funded £14.25m of the dividend with the remaining £1.25m coming from NATS (Services) Ltd, the non-regulated business providing airport air traffic services and consultancy.





ENDS

Plazbot 7th Dec 2013 22:57

A system on an aircraft goes unserviceable, the aircraft returns to earth and the thread is started and locked immediately. An ATC system goes US and the world has ended. Comparing the frequency of the two, ATC systems do pretty well. Well played Poms. Safety first.

1066 7th Dec 2013 23:00

I may have it wrong but is the trg facility at Whitely still updated to the same state as the main ops rooms at Swanwick? Any ATCOs care to comment?
1066

msjh 8th Dec 2013 07:24

Dividends to shareholders
 
If shareholders didn't get dividends, then they wouldn't invest in a company. That's what drives the economy. (And, incidentally, that's what pays most pensions) :ugh:


All times are GMT. The time now is 16:50.


Copyright © 2024 MH Sub I, LLC dba Internet Brands. All rights reserved. Use of this site indicates your consent to the Terms of Use.