Go Back  PPRuNe Forums > Flight Deck Forums > Rumours & News
Reload this Page >

Another 777 uncommanded engine rollback

Rumours & News Reporting Points that may affect our jobs or lives as professional pilots. Also, items that may be of interest to professional pilots.

Another 777 uncommanded engine rollback

Old 20th Dec 2008, 18:07
  #21 (permalink)  
 
Join Date: Dec 2000
Location: Arlington, Tx. US
Posts: 608
BS on the coke. Just a "Fate is the Hunter" troll.

The Sultan
The Sultan is offline  
Old 20th Dec 2008, 18:09
  #22 (permalink)  
 
Join Date: Dec 2008
Location: Burbleson AFB
Age: 51
Posts: 22
Regardless of similar airframe/engine combination, the fact that leaps out at me is that both these flights originated ay Shanghai.
Gen. Jack D Ripper is offline  
Old 20th Dec 2008, 18:12
  #23 (permalink)  
 
Join Date: Apr 2002
Location: FUBAR
Posts: 3,351
AnthonyGA, as a sceptical luddite whose knowledge of computers is limited to coming on here and giving my opinion, it is interesting to hear that someone who does understand a lot more, has the same misgivings as myself, but backed up by a knowledge of what is involved.
As I have said many times, if I see the tricks Windows Vista can pull ( and I am sure Microsoft have more money/resources invested than Boeing & Airbus combined) it doesn't fill me full of joy to imagine what lies within the systems of these aircraft for the future amusement of the crews.
Since I first expressed this view we have had BA, the Qantas 330 upset & the XL accident in PGF and now this. I may be an old fart, but my misgivings are not without grounds.
captplaystation is offline  
Old 20th Dec 2008, 19:23
  #24 (permalink)  
 
Join Date: Jan 2006
Location: UK
Posts: 130
Which is worse, a little knowledge of software or no knowledge?

and I am sure Microsoft have more money/resources invested than Boeing & Airbus combined
There is no comparison between the type of engineering Microsoft does (general purpose computers, consumer and business, huge number of operating modes, unimaginable combinations of hardware and software) and Boeing/Airbus does (single purpose, limited operating modes, known hardware/software).

AnthonyGA, sorry, you lead me to believe you might be in IT but know little about designing, engineering and testing embedded aviation software.

Verification and certification processes were designed for mechanical systems, not ones and zeroes.
Verification and certification of aviation software bears resemblance to mechanical systems processes in what way? Both Boeing and Airbus (and the vendors who supply software and hardware to them) operate with verification systems designed from the ground up,specifically for these applications.

Systems that mix software and hardware are still tested and verified as if everything were hardware, and that lets lots of potential bugs slip through the cracks
Please explain? Do you mean hardware without software - i.e. useless or are you saying mechanical systems, or avionics without software? Either way, please explain what you mean. I can see no parallels between testing airborne software and mechnicals.

No more likely to have an unknown bug in software than a design fault in mechnical systems - do a little research into how these systems are designed and engineered.

Would I rather travel in a mechnical aircraft or FBW? Actually, never crosses my mind. Both are demonstrably far safer than any other form of travel I use by far. And of course, safety just keeps getting better - including FBWs and other computer controlled a/c.

Finally, I'd be interested to hear how 3 different programs, written by entirely separate teams (with no knowledge of each other) running on different hardware can have the same bug? Design error yes (see above re mechnical systems) but bug? Any idea of the chance that 2 entirely separate systems can both get it wrong, at exactly the same time whilst the 3rd, entirely separate system gets it right?

Move along there, nothing to see...

Last edited by Simonta; 20th Dec 2008 at 19:54.
Simonta is offline  
Old 20th Dec 2008, 22:06
  #25 (permalink)  
 
Join Date: Jan 2003
Location: Manchester
Age: 42
Posts: 598
Regardless of similar airframe/engine combination, the fact that leaps out at me is that both these flights originated ay Shanghai.
Very good point.

But shhhhh, don't upset the Chinese...... I wonder why the BA investigation has gone so quiet....
Ex Cargo Clown is offline  
Old 20th Dec 2008, 23:27
  #26 (permalink)  
The Reverend
 
Join Date: Oct 1999
Location: Sydney,NSW,Australia
Posts: 2,020
That's stretching a bit Cargo Clown. I don't know how many 777s are amongst the hundreds of daily departures from Shanghai but if the engine rollbacks had anything to do with Shanghai fuel, there would have been more than two incidents of this nature.
HotDog is offline  
Old 21st Dec 2008, 00:04
  #27 (permalink)  
 
Join Date: Jan 2007
Location: Location: Location!
Posts: 1,991
Regardless of similar airframe/engine combination, the fact that leaps out at me is that both these flights originated ay Shanghai.

Sorry to rain on your parade, General Jack, but BA038 actually originated from what we must now call Beijing.

Jack
Union Jack is offline  
Old 21st Dec 2008, 01:00
  #28 (permalink)  
 
Join Date: Feb 2008
Location: In the Old Folks' Home
Posts: 397
Ignoring A Certain Fact

Many posters continue to try to implicate software but they completely ignore the fact that, in BA 038, in response to auto-throttle and power lever movement, both fuel metering valves opened fully, yet the engines did not respond. Does that not mean that there was no little or no fuel there to flow toward the nozzles? How then can software be implicated? Faulty software can cause a lot of problems, but the evidence does not point to software in the case of BA038. In fact, hard evidence seems to completely exonerate software.
Smilin_Ed is offline  
Old 21st Dec 2008, 02:20
  #29 (permalink)  
 
Join Date: Dec 2008
Location: Burbleson AFB
Age: 51
Posts: 22
Union Jack

You overstate the relevance of your correction ("...rain on your parade.." oh! purrlease!!!)

The fact is they were BOTH B777's, BOTH had RR Trent engines, BOTH originated in CHINA & BOTH had long cruises at high cruising levels in extremely cold temperatures.

I'm NOT saying that incorrect ratios of fuel additives was the cause, however, I don't think it was explored as thoroughly as it could have been during the investigation, certainly when compared to the scrutiny the airframe/engine hardware/software had to endure.
Gen. Jack D Ripper is offline  
Old 21st Dec 2008, 09:11
  #30 (permalink)  
 
Join Date: Sep 2007
Location: Paris, France
Posts: 351
Unfortunately, aviation software engineering isn't as different from other types of software engineering as many people in aviation would like to believe. The list of problems with aviation software is extremely long, although it is not widely publicized.

Most software fails because of design defects rather than bugs, although bugs are certainly important sources of problems. Aviation software is just as prone to design defects as any other type of software. Checking code against a spec will not prevent design defects. Having three versions of the code written by three different teams also will not prevent design defects. Even verifying bit patterns in firmware will not prevent design defects.

Design defects exist in mechanical systems, too. The problem is that defects in software manifest in a completely different way from defects in mechanical systems. Software systems usually have catastrophic modes of failure because of the disconnect that exists between bits and bytes and actual physical systems. Mechanical systems typically do not fail in catastrophic ways because of physical limitations that prevent failures from moving outside a certain envelope. Physical laws limit the magnitude and nature of mechanical failures. No such laws protect software.

A classic, simple example: Suppose you have a throttle lever that can mechanically travel between two stops, corresponding to idle and full power. A large number of different physical limitations prevent isolated failures from having dramatic effects. For instance, if the lower stop on the throttle fails, the worst that can happen is that the throttle will descend past idle—but since the engine is already at idle, that doesn't change much of anything. And if the upper stop fails, the throttle can move a bit past full power, but many other factors limit the effects of that: the throttle travel is only so far, the engines cannot run at two or three times full power no matter what the throttle setting, and so on. There's a lot of inevitable physical interaction that limits the effects of failures and design flaws. This interaction doesn't have to be built in by the designers, it's just a consequence of the laws of physics.

Now consider a "FBW" throttle. The throttle provides inputs to a computer, which then decides what setting to use for the actual engine throttle. The engine throttle settings are represented by a three-digit number, from 000 (idle) to 999 (full power). The throttle lever in the cockpit merely commands changes in throttle setting: pushing the throttle tells the computer to increase the throttle control number, and pulling the throttle back tells the computer to decrease the throttle control number.

Now you have a situation where the FBW can cause a catastrophic failure that would be impossible with the mechanical throttle. If the pilot advances the throttle repeatedly or very aggressively, the computer will continue to add to the throttle control number: 500, 501, 506, 600, and so on. What happens when the number reaches 999? Well, that's the problem. If the designers of the software haven't foreseen this in advance, adding one to 999 will produce 000. Which means that if the captain advances the throttle too far, the engines will be abruptly set to idle. That is catastrophic behavior that can cause a crash. And it WILL happen, UNLESS the designers foresee the possibility and program the software to not add anything to a throttle control number that is already at 999. The same problem exists if the pilots retard the throttles too far: they could go from 000 to 999.

In a mechanical system, there's no way for a throttle to snap back to idle if you push it past full power. In a FBW system, it's easy, and in fact it's guaranteed to happen unless the people who build the system specifically protect against the possibility.

That's why software is dangerous. Designers often overlook things, and the things they overlook can kill you a lot more easily than any mechanical failure could, because there are no laws of physics to limit the magnitude of a malfunction.

And while aviation is certainly more careful about software than many other fields, it's not nearly careful enough. I've been following the use of software in aviation for years, and it's grim. Way too much software is put in place with way too little thought, and periodically that causes problems, sometimes deadly problems.

The weird thing is that people tend to trust computers more than mechanical systems, when it should be the other way around. Yes, in theory, computers can do a better job, but as long as they have to depend on flawed software designed and written by human beings, they may actually be more dangerous than the mechanical systems they replace. It's interesting to see the blinders that people put on when they are looking at computer controlled systems.

They say that the rules of aviation are written in blood, but apparently that is only true if the blood is shed by mechanical systems. If software is at fault, people try to hide the problem, downplay the problem, and even resort to falsifying data to conceal the problem—they seem very unwilling to simply acknowledge and fix the problem, and equally unwilling to take responsibility for their design flaws and bugs. That kind of attitude might be more understandable when the only possible consequence is a day of bookkeeping lost, but I'm surprised to see it still in full force even when people die.
AnthonyGA is offline  
Old 21st Dec 2008, 09:49
  #31 (permalink)  
 
Join Date: Jul 2000
Location: Austria
Age: 59
Posts: 65
Anthony GA:
The weird thing is that people tend to trust computers more than mechanical systems, when it should be the other way around. Yes, in theory, computers can do a better job, but as long as they have to depend on flawed software designed and written by human beings, they may actually be more dangerous than the mechanical systems they replace.


Ever experienced a frozen Handle due to a struck elevator cable? Might make you feel different about mechanical systems being less flawed.
maxrpm is offline  
Old 21st Dec 2008, 11:03
  #32 (permalink)  
 
Join Date: Nov 2007
Location: between a rock and a hard place
Posts: 82
AnthonyGA

A very interesting and thought provoking post, I would be most intrigued to hear a counter argument, if anyone is so qualified, or indeed dares to give one.
scrivenger is offline  
Old 21st Dec 2008, 11:28
  #33 (permalink)  
 
Join Date: Mar 2004
Location: schermoney and left front seat
Age: 54
Posts: 2,369
From my experience I tend to say AnthonyGA has a point. The biggest airplane I ever flew yet is the Challenger 300, but that airplane has (had?) enough software generated problems and also problems that origin from the analog / digital interfaces. For example, we had erratic engine behaviour which was not reproducable, in the end we replaced the throttle quadrant (comes a s a complete unit) and thing worked as advertised again. The test and measurements where all within limits. I nowadays fly the Cessna C680 and on our airplane we sometimes don´t get the HP bleed from one side when anitIce is on. MX have checked everything and the condition is not reproduceable at will, it just happens sometimes. I found out that we could get it to work with changing fadec channels, however P&W and Cessna say its not a FADEC (aka software) problem. What is it then I ask? ADCs Rat probes etc have been checked....
And how do we deal with problems we can reset by cont/alt/del aka cycling the power? Hold item, MEL or just shrug shoulders? A wide field that needs a bit more attention IMO.

Regarding the certification of software, I´m not involved in that, but I ask myself how can they get stuff certified that acts sometimes really strangely, like the Honeywell Epic avionic suite installed in the C680...You are not allowed to use a lot of functions that originally have been okay to use and we get them back in little steps. The magical phase 5 software that is supposed to cure all the probs is delayed time and again, so methinks its a lot harder to find out and rule out software related problems than hardware.

Sorry for the OT.
His dudeness is offline  
Old 21st Dec 2008, 12:52
  #34 (permalink)  
 
Join Date: Jul 2007
Location: Switzerland
Age: 74
Posts: 98
dangerous software?

If some of the posters here would design computer controlled systems for aircraft, that would indeed be dangerous. Take a look at the example of the throttle encoding: angle encoders used for this kind of control are made in a way that precludes wrong readouts by design using an error tolerant coding technique (gray code or code with parity checking). They may not read out at all, or the link may be broken, but they most certainly do not go from 999 to 000. What rubbish. Nothing to do with software neither, by the way. There are other sensor techniques for command inputs like side sticks etc (like potentiometers or phase angel encoding), then they use redundant components or other fail safe designs.

We all agree that we would not trust an aircraft based on windows software, but you cannot compare an office system with an embedded and dedicated controller like FADEC or FMS or you name it, not be design and certainly not by price.

By the way, Captain Ernie Gann describes an uncommanded rollback on a flight from California to Hawaii in his book "fate is the hunter". It was no jet he was flying, it was a DC-4 running on 4 corncobs some 55 years before. Plenty of avgas on board. Now whatever the problem really was (Gann does not provide an answer, but resonance may have been a problem), a DC-4 was not known to be computer controlled. So such things did obviuosly happen without the help of software.
clearedtocross is offline  
Old 21st Dec 2008, 13:35
  #35 (permalink)  
 
Join Date: Nov 2007
Location: between a rock and a hard place
Posts: 82
So what we seem to be facing is the fact that aircraft, for at least 55 years, have had uncommanded engine rollback, and nobody is any the wiser. Now thats progress....

Surely everything that ever happens to something mechanical, be it instigated by a person or a computer MUST be correctly and fully monitored, otherwise there seems to be a total lack of inherent concern for safety with regard to that monitoring.
scrivenger is offline  
Old 21st Dec 2008, 14:00
  #36 (permalink)  
 
Join Date: Aug 2005
Location: East of eden
Age: 76
Posts: 147
I believe "AnthonyGA" was trying to make a point with his 000-999 analogy. I have a feeling he is well aware that is not how a FADEC system works. I also think both he and "His dudeness" have a point. Why did the E-190s (or whatever e jet it was) that USAirways introduced have such a dreadfull introduction? Because of the Magic..not the airframe. Look at the complex avionics packages being introduced into the fleets these days. After a few years in service where are we? Airframe? Same as certified. Software? -6,7 or -e,f etc. Certainly several iterations of the original installation. I fly/ have flown some of the most advanced avionics and FBW aircraft and I have no desire to go back to round dials and I just hate big yokes. I just think we need to be aware of the potential problems inherent with our reliance on software. When cont/alt/delete means a totally black airplane watch out!!
flown-it is offline  
Old 21st Dec 2008, 14:42
  #37 (permalink)  
 
Join Date: Sep 2007
Location: Paris, France
Posts: 351
I wasn't talking about the encoding of the throttle movements, which is mechanical, not software. I was giving an example of why software failures are so much more dangerous than mechanical failures. The fact that this was misunderstood helps to prove my point.

Apparently mechanical engineers have been humbled by centuries of mistakes and are at least moderately willing to accept that they might have made a mistake that could have put someone in danger. Not so for computer engineers, in my experience, and software engineers are the worst of the lot. Even when you walk them through the faulty code and show them their errors, they will continue to deny them. It's always someone else's fault.

It reminds me eerily of a certain airframe manufacturer and airline who immediately claimed that a chief test pilot was mentally disturbed after an accident, rather than admit that there was a software problem. Never mind that people died in the accident; that apparently was not important. (I've been unwilling to board their aircraft ever since, and from all reports their attitude has not changed.)

It worries me that more and more software is being put into service faster and faster with less and less verification in aviation. The things I hear about software-bloated avionics, for example, are identical to what I hear about desktop computer software, the only difference being that malfunctioning avionics kill people, whereas a malfunctioning spreadsheet usually does not. There should never, ever be a situation in which features included in an avionics package must be avoided after installation because of bugs. If the software were designed and tested properly, that would never happen—but obviously it is happening, which conclusively demonstrates that the software is indeed being put into real airplanes with life-threatening defects. And yet crews still are willing to fly on aircraft that depend on software that already contains proven defects. What makes them think that the defects discovered are the only defects lurking within? Typically, for every bug you find in software, there are dozens or hundreds of others that you haven't seen yet.

I don't understand the denial that I hear about this. Is it really preferable to die from the consequences of bad software than it is to admit that the software is bad? Or perhaps people insist that software is safe and adequately tested because it scares them to consider the very real likelihood that it isn't safe and isn't adequately tested.

Some of it is probably conditioning. Most people around today have grown up with computer systems that are incredibly unreliable, thanks to very hastily, poorly written software that has never been adequately designed or tested. They actually believe that this is normal, and that there's no other way for computers to work. They take this irrational belief into other domains where computers are increasingly used, including safety-of-life domains such as aviation. Problems that would lead them to riot and sue in a furious frenzy if they occurred in mechanical systems merely cause them to shrug their shoulders in software systems, as if software reliability depended on fate rather than hard science.

If it took NASA years and billions of dollars to produce very simple software running on very simple computers with a reasonable degree of reliability, do you really thing that software written in a month and modified every two months by underpaid Third World subcontractors working against deadlines, and tested based on obsolete and irrelevant certification criteria, is really going to be safer? Remember, whichever way you feel about it, you're betting your life on being correct.
AnthonyGA is offline  
Old 21st Dec 2008, 14:50
  #38 (permalink)  
 
Join Date: Jan 2007
Location: Location: Location!
Posts: 1,991
You overstate the relevance of your correction ("...rain on your parade.." oh! purrlease!!!)

Nice try, General, we all completely understand that:

The fact is they were BOTH B777's, BOTH had RR Trent engines, BOTH originated in CHINA & BOTH had long cruises at high cruising levels in extremely cold temperatures.


which are all indeed very relevant, but the fact is that the point you actually tried to make was:

Regardless of similar airframe/engine combination (my italics), the fact that leaps out at me is that both these flights originated ay (sic) Shanghai

Some leap since even a presumed Spam should know that Shanghai is not exactly a suburb of Beijing!

Now, just let the real experts get on with it

Jack

PS Congratulations on your promotion ......
Union Jack is offline  
Old 21st Dec 2008, 20:55
  #39 (permalink)  
 
Join Date: Jun 2003
Location: EuroGA.org
Posts: 13,786
Finally, I'd be interested to hear how 3 different programs, written by entirely separate teams (with no knowledge of each other) running on different hardware can have the same bug? Design error yes (see above re mechnical systems) but bug? Any idea of the chance that 2 entirely separate systems can both get it wrong, at exactly the same time whilst the 3rd, entirely separate system gets it right?
I don't think you do software either!

In the real world, programmers come from the same "school" where they learnt the same algorithms, they read the same internet programmer forums, they pick up bits of the same open-source source code, lift the same algorithms off the internet, they make the same mistakes generally (stray pointers etc etc etc), so while having three different teams write some code to the same overall spec will protect against a lot of problems, it will fall far short of protecting against all.

And in many ways, the overall system spec will plant the seeds for the three teams making the same mistake in the same place.

I've been doing hardware/software for 30 years. One can get it damn good, and trivial bits of software can be made totally (and provably) reliable, but as soon as it departs from being trivial, the reliability is no longer assured, and can no longer be proven. One can still make it damn good if one is really careful, even to the extent where no customer ever finds a bug over say 10 years in a 24/7 industrial application, and I have achieved that with a number of products, even writing in assembler, but you still never really "know" what is around the corner - because most usage patterns are similar and test only a small part of the functionality.

The other thing is that the "box" which compares the outputs of the three systems will be the new single point of failure...

I have read the entire BA038 thread, and the intermediate AAIB reports, and yes I agree it doesn't look like a control system issue because the fuel control valves were found to be fully open. Plus the cavitation damage, etc.

But I wonder how reliable is the position encoder / data acquisition on the fuel control valve. Sensors can and do fail. OK, one would not get two failing on both engines at the same time, but there is an awful lot of software between those sensors and the recorded data on the valve positions. Many millions of instructions executed to collect and store that data.
IO540 is offline  
Old 22nd Dec 2008, 00:09
  #40 (permalink)  
 
Join Date: Mar 2002
Location: Florida
Posts: 5,336
I don't understand why all i8rrelevant talk about computer knowledge when we still don't have clue to what precipitated the rollback.Most of the rollbacks that I have come across start with mechanical deterioration and it's the computer that is tweaked to accomodate the problem to fix some of the rollback issues

So have any new facts surfaced like commanded fuel metering vs a timeline
lomapaseo is offline  

Thread Tools
Search this Thread

Contact Us - Archive - Advertising - Cookie Policy - Privacy Statement - Terms of Service - Do Not Sell My Personal Information

Copyright © 2018 MH Sub I, LLC dba Internet Brands. All rights reserved. Use of this site indicates your consent to the Terms of Use.