Power Failure In Egpxacc

[atco-matic]

Is EGPX prestwick/ scottish? CFMU says there was a power failure this morning for ten minutes at 0850, but normal ops now apparently... any more info anybody?

FOLLOWING A TOTAL POWER FAILURE IN EGPXACC REGULATIONS WITH
REDUCEDRATE ARE APPIED IN SEVERAL SECTORS.
.
.
FMD STAFF ARE CONTINUOUSLY MONITORING THE SITUATION AND
WILL ADVISE OF ANY POSSIBLE IMPROVEMENTS.
.
IN THE MEANTIME OPERATORS ARE STRONGLY REQUESTED NOT TO
TELEPHONE FMD HELPDESK CONCERNING THIS PROBLEM

[1261]

Yes there was; no problems; power restored (fairly) quickly with no drama!

[PH-UKU]

I heard that all radar displays, all phonelines and all mains frequencies went 'plink'. Is that correct ? Wouldn't like that happening to me at FL350.

Did the electric flushing toilets still work ?

[NATS_Not_Funny]

Problems with UPS switching, then problems reverting as I understand. Less that 500 min delays in total and no safety problems.
Lots of pressure on airport ATC as well. Just the sort of situation that shows how well everyone can work towards a common goal.

Well done to everyone for the professionalism shown :D

[PH-UKU]

No safety problems ? What's the controllers viewpoint on this ? I would have thought losing radar and main comms was pretty hair-raising. Was this an equipment fault or an engineers cock-up ?

[Lost_luggage34]

This smells of lack of investment again. There is virtually no reason why a UPS system supporting such a critical datacentre should fail in this way.

I do not know the full facts but, having read what I have seen written here, along with information from other sources, I am very shocked.

[Scott Voigt]

I've been through a true power failiure where you have nothing. That is by any standard NO FUN! If anyone says that safety was not comprimised they are telling a rather large lie... Unless of course there is no traffic... If you are using radar separation and then have no radar you have to go to non radar and you have a lot of what we call here in the states, "deals". Now, like I said, if you don't have much traffic, then not much of a problem, but if you are working departures, arrivals and a lot of crossing traffic, that is a bit different... Can't see or talk for ten minutes, that is no fun...

regards

[NATS_Not_Funny]

What I meant was that as far as I know there were no crashes, near misses or other reportable safety incidents between planes as a direct result of this outage.

I was trying to point out how well everyone had handled a very bad situation.

PH-UKU
Was this an equipment fault or an engineers cock-up ?
Does it make a difference to you what the root cause was?
Do you need someone to blame?

I personaly know some of the story, but not enough to comment on why it happened. I would suggest others in the same position wait until the facts are known before blaming it on management :)

"A lie can run round the world before the truth has it's boots on"
Terry Pratchett

[Bigears]

Airvent, yes and no- I offered a green PC screen (remember the very early ones?) for £10, but it wasn't taken up :D

[Aunt Rimmer]

<1261> "no problems, no drama"

Erm, maybe not 'on the ground', but as one who was on sector at the time I can assure you it was scary - but of course we were told it could never happen. Lights flicker, big 'clunk', silence as aircon stops and the radar screens go blank blank blank. Frequency dead and telephone lines dead. All faces look left and right in amazement, colour drains into trousers. Pause for breath. Grab for emergency handset, select both frequencies in use, try to use R/T with handset clamped to left ear, write with right hand, no hand spare to try phone lines, no standby phone available for 10 minutes until programmed by Tels. Resorted to asking SH360 on one freq to call EGAA on another to tell them we had crashed - how's that for a backup? Offload tfc to adjacent units. Radar came back after about 6 mins (that's about 50 miles at cruise speed), but it felt like 20, phones were back similar time. But you are disorientated for a good while and need to get radars reset while still keeping on top of the tfc.....it all takes time.

<NNF> "Less that 500 min delays in total and no safety problems.
Just the sort of situation that shows how well everyone can work towards a common goal." :mad:


NNF, I think you need a reality check. Also let me know when you are at work cos' it scares me if these are your yardsticks. No safety problems ? WHAT PLANET ARE YOU ON? Just the sort of situation that I hope never to experience again. One of the most stomach-churning moments for me in 16 years controlling - but of course it could never happen..... if you want to identify yourself and buy me and the rest of the watch a beer to explain your logic then feel free to email.

Thank God it wasn't busier. If that had happened 90 mins earlier in the middle of the morning rush it would have been chaotic. As it was there were probably about 70ac in the air at the time.

But at least all the computers in the offices stayed on, and the coffee machine in the canteen didn't go off :p

Amazed that it stayed out of the news. Read the class action 1261 when it comes out.

As a learning point, nothing in Emergency Training prepares you for the reality. My advice is have a practice on the nightshifts using handset and emergency phones to at least see how disorientating it is.

[Pheasant Plucker]

'Aunt Rimmer' !! - where did you get that name from?? :D

Can I just concur with what my esteemed colleague has just typed.

'No safety problem', 'no drama'??

On the sector that I was doing at the time, I can assure you, it was purely fortuitous that separation wasn't lost, despite my best efforts.

By definition, if you are working in a radar environment and your main tool for separation purposes (...the radar display), goes tits up, you automatically have a serious safety issue.

If this had happened an hour earlier, there is no doubt in my mind that a serious loss of separation would have occured.

[Aunt Rimmer]

I have a red dwarf :D

[PH-UKU]

NATS_not_Phunny - your initial tone was very patronising. If I were an ATCO I'd resent your 'no problemo' 'eachy-peachy' attitude. Words are cheap from sitting behind a desk, but same as in the cockpit YOUR ass/licence/sanity ain't on the line in that brown trouser moment. And I sure as hell want to know if my engine fire is caused by human or mechanical error.......so ?

And as far as I'm aware no-one has blamed management, so why the defensive stance ?

Aunt Rimmer - hats off to you all for keeping us pests apart, it sounds pretty stressful, but thank you for sharing. I think you all deserve an extra week off for that - deduct it from the guilty bozo ! Off for a sunshine sector tomorrow - Ciao.:cool:

[Pheasant Plucker]

Thought Aunt Rimmer was just a description of your hobby.

Does she enjoy it?

Does your red dwarf get jealous?

I think we should be told.

Night Night

[Cuddles]

Look for a frazzled man holding a drill.

IF the building of the NSC had gone ahead, prime suspect would have been a halfwit wielding a JCB.

[NATS_Not_Funny]

PH-UKU, Aunt Rimmer
Next time I think about posting a quick compliment to my colleagues I'll phone my lawyer, spin doctor, analyst, and 7 specialist advisors. That way I might not get my arse chewed off for not wording it right.

Subject closed.:(

[Minesapint]

For those that remember - or not - this happened at LATCC in - I think - 1992. It took days to get the smell out of the ops room! Difference was that this one took NAS with it. In this case the guys at LTCC saved NAS by removing the Scottish centre from its sphere of interest!

By the way - the "its not safe", "it was scary" is for ATCO's and pilots to decide. NOT for us support people. :rolleyes:

[Aunt Rimmer]

NNNF - "Next time I think about posting a quick compliment to my colleagues I'll phone my lawyer, spin doctor, analyst, and 7 specialist advisors. That way I might not get my arse chewed off for not wording it right.

Subject closed."

Compliment accepted - albeit not as graciously as you might have liked. Still owe me+PP a pint, though :p

And why run away with your tail between your legs, just when it's getting interesting ? Heat + kitchen spring to mind. Mind you we're not allowed to use the kitchens without an engineer present, in case we hurt ourselves ...;)

[commseng]

Haven't checked the location you are talking about. But I think you will find that all ATC providers in the UK have RT & Phones which are totally battery supported all the way to the controllers suite. But radar can be a problem, you can't run it on a PP3 battery! But most people have standby generators & UPSs.

[Crappy Headset]

I was also plugged in when it all went quiet and it wasn't good for the old ticker.

General manager observed standing in the ops room just after it happened but no words of thanks, apology or any form of explanation forthcoming yet.

I agree with previous posts - if this had happened during the morning rush it would have been horrendous.

A similar scenario when no back-up power kicked in happened years ago and we were told it wouldn't happen again!!! I just hope I'm not plugged in when it does happen again.

[1261]

For those commenting on my original post; I can assure you that it was intended for the "unscrupulous" reading the forums immediately after the event!

I too was plugged in when it happened, after all. Luck for us we only use Lowther for SSR. Even then, it's not often that I get calls from aircraft in the Shap/DCS area, and our primary doesn't reach that far....

[Dungeon dweller]

:confused: :confused:
And if the press had published it, it would have been the fault of Swanwick as they would have finished the article something like...."NATS, whose £600m ATC Centre at Swanwick continues to have problems..." You just can't win with the press.

I would have thought the lessons of the LATCC power problems would have been learnt across the company.. Hey ho! I've just spotted another squadron of flying porkers....:mad: :mad: :mad:

[danceswithsheep]

Aunt Rimmer v Pheasant Plucker.........sounds like wedding bells in the near future.
I have to be invited though as I agree with the comments posted. It takes an ass to think all is ok with a total power failure. We worked hard just to get power back to be hastled by the budget because that put them behind!!!
We (ATC and Pilots) need to get together a little more for some understanding. I welcome pilots to Prestwick Centre for a visit and if any want to give Fam Flights to any warm climates, thats a good deal!
In all seriousness, we have to work together. We can do better and from experiance, if we at Atc are happy and fulfilled, naturally we work better. Think about that one.

[WetFeet]

Power outage believed caused by unauthoriesd switching off of the UPS by a contractor whilst batteries were isolated.

By the way, GM thanked watch management, did they not pass it on?

[danceswithsheep]

Ripped apart when you make a mistake............no personal touch when you get them out of the bog. Do you really think we are being treated like well respected staff, or the cheap help we get the feeling we are. One day management will figure it out that without us, they have no job either!
Just pay us what we are worth or at least come in, in person and thank us for a difficult situation dealt with as proffesionally as we could. Not through a Watch Manager. :eek:

[NERC Dweller]

DD wrote

I would have thought the lessons of the LATCC power problems would have been learnt across the company.

I think they have been learnt. NERC may have its problems but power is not one of them.

No 2 Workstations on a Sector Suite run from the same Power Supply. Server pairs are powered from a different supply. Even if an UPS was to fail, the backup UPS would 'step-in' without interrupting the power.

I can't guarantee that Power will never fail totally but its the closest it can go

I believe NSC was going to have a similar system before it was cancelled.

Why not retrofit into LTCC/MACC/ScACC? I don't think it is physically possible while in Operational Service (too much kit has to be powered off to do the wiring)