PPRuNe Forums - View Single Post - Another 777 uncommanded engine rollback
View Single Post
Old 21st Dec 2008, 14:42
  #37 (permalink)  
AnthonyGA
 
Join Date: Sep 2007
Location: Paris, France
Posts: 350
Likes: 0
Received 0 Likes on 0 Posts
I wasn't talking about the encoding of the throttle movements, which is mechanical, not software. I was giving an example of why software failures are so much more dangerous than mechanical failures. The fact that this was misunderstood helps to prove my point.

Apparently mechanical engineers have been humbled by centuries of mistakes and are at least moderately willing to accept that they might have made a mistake that could have put someone in danger. Not so for computer engineers, in my experience, and software engineers are the worst of the lot. Even when you walk them through the faulty code and show them their errors, they will continue to deny them. It's always someone else's fault.

It reminds me eerily of a certain airframe manufacturer and airline who immediately claimed that a chief test pilot was mentally disturbed after an accident, rather than admit that there was a software problem. Never mind that people died in the accident; that apparently was not important. (I've been unwilling to board their aircraft ever since, and from all reports their attitude has not changed.)

It worries me that more and more software is being put into service faster and faster with less and less verification in aviation. The things I hear about software-bloated avionics, for example, are identical to what I hear about desktop computer software, the only difference being that malfunctioning avionics kill people, whereas a malfunctioning spreadsheet usually does not. There should never, ever be a situation in which features included in an avionics package must be avoided after installation because of bugs. If the software were designed and tested properly, that would never happen—but obviously it is happening, which conclusively demonstrates that the software is indeed being put into real airplanes with life-threatening defects. And yet crews still are willing to fly on aircraft that depend on software that already contains proven defects. What makes them think that the defects discovered are the only defects lurking within? Typically, for every bug you find in software, there are dozens or hundreds of others that you haven't seen yet.

I don't understand the denial that I hear about this. Is it really preferable to die from the consequences of bad software than it is to admit that the software is bad? Or perhaps people insist that software is safe and adequately tested because it scares them to consider the very real likelihood that it isn't safe and isn't adequately tested.

Some of it is probably conditioning. Most people around today have grown up with computer systems that are incredibly unreliable, thanks to very hastily, poorly written software that has never been adequately designed or tested. They actually believe that this is normal, and that there's no other way for computers to work. They take this irrational belief into other domains where computers are increasingly used, including safety-of-life domains such as aviation. Problems that would lead them to riot and sue in a furious frenzy if they occurred in mechanical systems merely cause them to shrug their shoulders in software systems, as if software reliability depended on fate rather than hard science.

If it took NASA years and billions of dollars to produce very simple software running on very simple computers with a reasonable degree of reliability, do you really thing that software written in a month and modified every two months by underpaid Third World subcontractors working against deadlines, and tested based on obsolete and irrelevant certification criteria, is really going to be safer? Remember, whichever way you feel about it, you're betting your life on being correct.
AnthonyGA is offline