Go Back  PPRuNe Forums > Flight Deck Forums > Rumours & News
Reload this Page >

Ethiopian airliner down in Africa

Rumours & News Reporting Points that may affect our jobs or lives as professional pilots. Also, items that may be of interest to professional pilots.

Ethiopian airliner down in Africa

Old 8th Apr 2019, 14:22
  #3621 (permalink)  
 
Join Date: Jul 2005
Location: btw SAMAR and TOSPA
Posts: 565
Originally Posted by kilomikedelta View Post
perhaps if the software and sensor device drivers were written in assembler by programmers who understood the hardware inter-relationships,these accidents could have been avoided.
I suppose machine language and assembler are foreign languages these days with few practitioners but the MBA's want things done cheap and dirty especially if they can outsource if offshore.
This is a serious system design matter. MCAS does its job as planned. It is just the wrong job. There is no need for assembler to speed anything up. It makes code harder readable, less maintainable and certifyable ... oops.
threemiles is offline  
Old 8th Apr 2019, 14:25
  #3622 (permalink)  
 
Join Date: Jul 2007
Location: Germany
Posts: 533
Originally Posted by Capn Bloggs View Post
Consider AF447. The design team thought that an aeroplane could not be flying below 60KIAS, so they inhibited the stall warning... We know how that ended.
In hindsight that is probably a shortcoming of the risk and hazard assessment of the ADIRUs of the A330, yes. But the rationale for regarding all air data invalid at indicated air speed below 60 knots is that it was simply not known how the aircraft would behave in that regime. And it is sometimes a better idea (although that, too, requires careful evaluation and assessment) to flag all data as invalid (or not deliver it at all) rather than deliver data wich may be wrong.

As far as I recall the A3330 was not flight tested into a full stall, and very few other ways of flying that slow can be envisioned.

On the other hand it is not clear that the "reverse logic" (pitch down in a deep stall: stall warning comes on, raise the nose again: stall warning stops) of the stall warning at very high angles of attack and very low indicated air speeds was a causal factor in the accident: There are some indications that the crew did not only not react to the stall warning, but that they literally did not perceive it. It was lost to cognitive overload.

It could be possible to have an AOA of 75° and appreciate some assistance from the machine (MCAS) to keep the nose from getting there in the first place. The F up was making it reliant on only one sensor.
While the absolute value may be valid (although 75° is probably a bit of a stretch, even AF447 never even reached 50°. but the 25° or so from Lion Air is clearly in an attainable range), some things can also be deduced about the validity by evaluating rate of change, or cross-checking with other values, such as static and total pressures, vertical speed, pitch angle, acceleration, etc., even without cross-checking with the other-side AoA.


Bernd
bsieker is offline  
Old 8th Apr 2019, 14:34
  #3623 (permalink)  
 
Join Date: Jul 2005
Location: btw SAMAR and TOSPA
Posts: 565
Originally Posted by DaveReidUK View Post
The aircraft was more-or-less level and only just above Vmo at the point where those two brief applications of trim occurred.

The widely quoted 500 kts was subsequent to that, the result of the last episode of MCAS AND trim kicking in and the unrecoverable descent.
Also, they were in a 30 degrees bank turn when things started to really go wrong. Almost no pitch, 340 kts, no ability to pull or trim ANU, how can that work out? Before the bank had increased slowly over 2 to 3 minutes for whatever reason.
Then they were flying at 10000 feet, inside a MSA sector of 14000 ft, heading towards range with a peak of 11000 MSL, with little to no climb rate. Albeit in VMC, this alone is brutal.

Last edited by threemiles; 8th Apr 2019 at 14:45.
threemiles is offline  
Old 8th Apr 2019, 14:39
  #3624 (permalink)  
 
Join Date: Jul 2007
Location: Germany
Posts: 533
Originally Posted by kilomikedelta View Post
perhaps if the software and sensor device drivers were written in assembler [...]
That is possibly the worst suggestion so far.

Assembly code is almost impossible to analyse for correctness in any meaningful way. It is far better (and provably so) to write in a well-specified (i. e. not C) language, prove the source code correct (for which scalable and practical techniques exist today), or define and prove correct a finite state machine and have code generated from it. That still leaves one with a need to have reasonable confidence in the compiler, but in many cases the service history for the most-used language core, and, in some recent cases, formally verified compilers, take care of that.

Just because you have one hero programmer who claims to have done it "Right" in assembly does not help you in any way because you need to demonstrate that it does what it is supposed to do (reliability), and never does what it is not supposed to do (safety), and ideally also never fails (availabilitiy). And this cannot be demonstrated by testing alone to the extremely high requirements needed in aviation. Assembly and machine code are avoided like the plague in safety-critical programming, and rightly so. Where some parts require it, extreme care must be taken to get it right, and the amount must be kept to a minimum.

Besides, as threemiles has pointed out, the implementation is not the problem (as far as we can tell, it may be flawless), but the specification. "Working as specified" can also mean that it did the wrong thing.

Bernd

Last edited by bsieker; 8th Apr 2019 at 14:42. Reason: typo
bsieker is offline  
Old 8th Apr 2019, 14:48
  #3625 (permalink)  
 
Join Date: Feb 2015
Location: New Hampshire
Posts: 151
Originally Posted by kilomikedelta View Post
perhaps if the software and sensor device drivers were written in assembler by programmers who understood the hardware inter-relationships,these accidents could have been avoided.
I suppose machine language and assembler are foreign languages these days with few practitioners but the MBA's want things done cheap and dirty especially if they can outsource if offshore.
Having programmed in machine language, I would NOT recommend it. It would be very difficult to reach the level of confidence for direct machine code (or even assembly) that would be required for this software.

Just to be clear, we are talking about software that must not allow the MAX to do a mid-flight back flip and/or break up while also not dooming the plane to a high-speed nose dive. And if it fails and is disabled, the pilot may not be able to act fast enough to recover. It's hard to image a software component on an ATP flight that is more critical.

The sequence would be: requirements, requirements review, design, design review against the requirements, test development based on the design, test procedure review, coding, code review, code testing. This requires code that can be examined by several team members with no chance of misinterpretation.

So what is needed it a well exercised development environment - one that's been around for several years and has been very widely used - with a good reputation and version release notes that reflect a solid tool.
.Scott is offline  
Old 8th Apr 2019, 14:55
  #3626 (permalink)  
 
Join Date: Nov 2018
Location: madrid
Posts: 47
Originally Posted by bsieker View Post
In general, the industry does, and overall does a very good job of it. Input value filtering is normal, and even the A330 that nose-dived in cruise had computers which did. It was done inappropriately, but most of the time the erroneous values were rejected.

The problem here lies much earlier, in specifying the requirements for MCAS. More specifically, the failure modes and the severity and likelihood of each were not properly analysed.



Au contraire. Aviation is the industry which mandates appropriate techniques. They are well-known, and used.

Add-on systems that bring airliners back into compliance, which are not by themselves aerodynamically completely compliant to regulations, are literally as old as jet airliners themselves. Many types have stick-nudgers or stick-pushers, and they work fine, and are perfectly sensible to use. But that does not mean one can skip due diligence in developing them, which includes a thorough risk and hazard assessment.

Bernd
I know some plane computers are properly built and programmed and that part of the industry works ok. That's exactly what I meant with 《we know how to do it》.

What I meant is that there is a 《all or nothing》spirit that doesn't quite cut it. Either is a 8 year development with millions of man hours on it ,or a terrific patch that looks like done overnight. No middle ground.

I'm all in for a cheap patch. The cheapest possible, no need to doom the plane and start over. But if the computers in the plane have no room for improvement, a software patch in 1980s style is not going to be enough. We need sanity checks, cross sensor checks and history checks on ALL sensors. Not only AOA.

Precisely, no software patch will fix the issue of mcas being useless for real life (too slow).
ecto1 is offline  
Old 8th Apr 2019, 14:56
  #3627 (permalink)  
 
Join Date: Apr 2014
Location: YARM
Age: 70
Posts: 125
Originally Posted by CurtainTwitcher View Post
. . .
They faced a continuous cacophony of noise (stick shaker), that has been shown by anecdotal reports in this thread to induce tunnel vision and difficulty doing ANY task, including just flying the aircraft. Two crew were unable to cope, the third had assistance from an additional pilot.

Perhaps as the lawyers are debating and dissecting second by second details, they should do it with the continuous Stick Shaker loop in the background.
Indeed.

unworry is offline  
Old 8th Apr 2019, 15:01
  #3628 (permalink)  
 
Join Date: Feb 2015
Location: New Hampshire
Posts: 151
Originally Posted by bsieker View Post
It is far better (and provably so) to write in a well-specified (i. e. not C) language, prove the source code correct (for which scalable and practical techniques exist today), or define and prove correct a finite state machine and have code generated from it.
I doubt that they used C or C++, but given appropriate coding standards, I do not have problems with either. In fact, a major issue with keeping the code in alignment with an understandable design is in what kind of awkward and extraneous semantics the programmers needs to go through to getting the code to do what they want. Languages like C# are advertised as "safer" because they prevent certain types of errors - like memory management. But they add a major layer of code that is unrelated to the final functionality. That makes the really important parts of the code diffused and harder to review.

When I say "appropriate coding standards", the issue is reviewability. So using the C++ support for object oriented coding is good. Using polymorphism or the "virtual" keyword is avoidable - and for an application like this, should be avoided.
.Scott is offline  
Old 8th Apr 2019, 15:11
  #3629 (permalink)  
 
Join Date: Feb 2015
Location: New Hampshire
Posts: 151
Originally Posted by ecto1 View Post
I'm all in for a cheap patch. The cheapest possible, no need to doom the plane and start over. But if the computers in the plane have no room for improvement, a software patch in 1980s style is not going to be enough. We need sanity checks, cross sensor checks and history checks on ALL sensors. Not only AOA.

Precisely, no software patch will fix the issue of mcas being useless for real life (too slow).
Interesting. I guess somewhere in this thread is a revelation that the MCAS hardware was too slow - I hadn't heard that. It it's from the 1980's, where they using Ada? That was one of those "safe" languages - but may come with a lot of overhead. If it had been coded in C, there would have been no practical problem with the machine speed.

If I was the FAA, I would not allow the plane to fly without having simulators available that fully simulate operations where the MCAS can work and fail - and then require that the pilots go through the different failure modes before visiting the MAX cockpit. If the MCAS hardware needs to be replaced, replace it.

I think that if Boeing tackles this problem correctly, they could have the planes flying safely in about four months. Or they can go for Micky Mouse fixes and risk either an ineffective fix or a recovery timeline that goes well beyond four months.
.Scott is offline  
Old 8th Apr 2019, 15:17
  #3630 (permalink)  

Only half a speed-brake
 
Join Date: Apr 2003
Location: Commuting home
Age: 42
Posts: 3,037
Originally Posted by bsieker View Post
While the absolute value may be valid (although 75° is probably a bit of a stretch, even AF447 never even reached 50°. but the 25° or so from Lion Air is clearly in an attainable range), some things can also be deduced about the validity by evaluating rate of change, or cross-checking with other values, such as static and total pressures, vertical speed, pitch angle, acceleration, etc., even without cross-checking with the other-side AoA.
MelVic's original remark read to me that accepting 75° raises questions how was the input validation done if at all.
FlightDetent is offline  
Old 8th Apr 2019, 15:36
  #3631 (permalink)  
 
Join Date: Sep 2007
Location: Europe
Age: 43
Posts: 30
Hi,

can anyone pls explain to me, why they had an IAS disagree with just an AOA sensor fault.

I thought the pitot/static system is independent...or is there some „mixing of data“ going on in the ADIRUs?

thx
KRH270/12 is offline  
Old 8th Apr 2019, 15:42
  #3632 (permalink)  
 
Join Date: Apr 2008
Location: Paris
Age: 70
Posts: 275
Why not just disable MCAS, leave the pilots to trim as usual, and retrain them a bit to deal with the feel of the plane at a high AoA?
Zero software to write, and no new bugs introduced.
Because as they say in the industry, new features always means new bugs.


Edmund
edmundronald is offline  
Old 8th Apr 2019, 15:45
  #3633 (permalink)  
 
Join Date: Jul 2007
Location: Germany
Posts: 533
Originally Posted by ecto1 View Post
What I meant is that there is a 《all or nothing》spirit that doesn't quite cut it. Either is a 8 year development with millions of man hours on it ,or a terrific patch that looks like done overnight. No middle ground.
I'm not sure what gave you that idea. There are various levels of criticality for different pieces of software.

However, there are bigger issues at stake. MCAS has the potential, without quick and correct intervention by the crew, to cause a catastrophic outcome ("catastrophic" is not just a fancy term, it is quite well defined in certification specifications.) Therefore software for systems with such severe possible consequences, need to be developed to particularly stringent standards of requirements specification, analysis, coding practices, planning, documentation, verification, etc. That is what some people here mean when they refer to "DAL A" or "Level A": That is the most stringent category for safety-critical software in airborne systems, as defined in DO-178C (or ED-12C in Europe, which is excatly the same standard):

Level A: Software whose anomalous behavior, as shown by the system safety assessment process, would cause or contribute to a failure of system function resulting in a catastrophic failure condition for the aircraft.
As we now know, MCAS is just such a system. So, as others have pointed out repeatedly, at least in hindsight, anything less than Level A is not appropriate.

Objectives that need to be demonstrated include things like
  • High-level requirements are accurate and consistent.
  • Low-level requirements are verifiable.
  • Software architecture is verifiable.
  • Source Code is verifiable.
  • Source Code is traceable to low-level requirements.
  • Source Code is accurate and consistent.
  • High-level requirements are accurate and consistent.
  • Low-level requirements are traceable to high-level requirements.
And many more. That is "the cheapest fix possible". It's not cheap, but it's doable in significantly less than 8 years for a company which has the procedures in place, which Boeing does.


Bernd
bsieker is offline  
Old 8th Apr 2019, 15:47
  #3634 (permalink)  

Only half a speed-brake
 
Join Date: Apr 2003
Location: Commuting home
Age: 42
Posts: 3,037
1) CAS is derived from pitots and static using an adjustment for AoA.

2) To disable MCAS: physically workable suggestion but knowingly failing a certification requirement is not an option.
FlightDetent is offline  
Old 8th Apr 2019, 15:59
  #3635 (permalink)  
 
Join Date: Sep 2018
Location: Laredo, TX
Posts: 123
Originally Posted by edmundronald View Post
Why not just disable MCAS, leave the pilots to trim as usual, and retrain them a bit to deal with the feel of the plane at a high AoA?
Zero software to write, and no new bugs introduced.
Because as they say in the industry, new features always means new bugs.


Edmund
Agreed, waive the Part 25 criteria and show the handling change in the sim. It would be nice to know what that envelope is. I'm surprised 737 pilots, pre the final incident, were happy flying an aircraft where if they were unlucky enough to apply the Emergency AD, they then would be at risk of encountering the envelope without MCAS and Boeing/FAA do not even warn about it in the AD.
jimtx is offline  
Old 8th Apr 2019, 16:12
  #3636 (permalink)  
 
Join Date: May 2008
Location: denmark
Posts: 5
Originally Posted by bsieker View Post
That is possibly the worst suggestion so far.

Assembly code is almost impossible to analyse for correctness in any meaningful way. It is far better (and provably so) to write in a well-specified (i. e. not C) language, prove the source code correct (for which scalable and practical techniques exist today), or define and prove correct a finite state machine and have code generated from it.
I would assume that it is either SCADE, or Simulink.
Both tools generate C code, but with SCADE you don't have to inspect the C once the tool chain have been qualified. (Considered the safest alternative of the two)
SCADE is used by Airbus (Both French products), and Boeing might use Simulink.
But this is irrelevant for this discussion since the fault is in the specification, not the software.
HighWind is offline  
Old 8th Apr 2019, 16:26
  #3637 (permalink)  
 
Join Date: Aug 2005
Location: EDLB
Posts: 205
Originally Posted by .Scott View Post
I think that if Boeing tackles this problem correctly, they could have the planes flying safely in about four months.
My guesimate is more a year if they have already started and they are lucky. Can easily be more. A Level A software piece need to run on a hardware worth that level. Don’t think that they have it in the 737 so they will need an additional box.
So I don’t think that the MAXes in storage will hit the friendly skies anytime soon.

Or does anyone believe that the FAA will stick their neck out and allow the next “quick fix”.
EDLB is offline  
Old 8th Apr 2019, 16:35
  #3638 (permalink)  
 
Join Date: Jan 2013
Location: UK
Age: 59
Posts: 36
There are a lot of confident and ridiculously detailed statements about the softare use din MCAS even the language used to code it. None of thsi is relevant. It is clear that the root is a system design error compounded by a failure of hazard and failure analysis and the regulatory/certification process.

I can see quick fixes which address the deficiencies in the way MCAS responds to erroneous inputs which from a functional point of view make the behaviour safe. I can't see a quick fix which address the wider hazard/failure analysis and regulatory concerns.

I write safety related software but discussing which language or tools to use to develop SW when the fundamental concept is flawed make no sense. The best solution to a safety hazard is intrinsic - remove the hazard, in this case that means aerodynamic changes and I assume that won't happen but it should have been thought about at the design stage. If intrinisc safety is not possible then a functional safety like MCAS is possible but the consequences of failures must be considered and controlled and any additional hazards by introducing the functional safety sub-system must be considered. I assume this was done, at least in the formal sense but it seems to have been done inexplicably poorly. This was not a case of a complicated combination of unlikely events but an entirely forseeable concsequence of a single failure. IF MCAS is retained it has to be designed and developed appropriately given the impact of it failing and that is not going to happen very quickly.

Ther eis no evidence I am aware of that the software did anything other than what it was intended and specified to do. Even the software specification was not really the eproblem. The problem was the overall system design and the safety analysis behind it. It is actually quite shocking and shoud result in a very deep analysis of both the development and regulatory processes.
PiggyBack is offline  
Old 8th Apr 2019, 16:42
  #3639 (permalink)  
 
Join Date: Jun 2010
Location: On the ground too often
Age: 45
Posts: 129
Originally Posted by bsieker View Post
In hindsight that is probably a shortcoming of the risk and hazard assessment of the ADIRUs of the A330, yes. But the rationale for regarding all air data invalid at indicated air speed below 60 knots is that it was simply not known how the aircraft would behave in that regime.
I think you are either missing the point or perhaps I do not fully understand what you mean. In real life reality an A330 could not be at or below 60 knots kias other than either with the wheels on the ground or during some quite spectacular upset - and even then it would last for maybe a few seconds. I think the only sensible rationale was to reject an indicated airspeed of 60kts for an airliner in cruise many tens of thousands of feet above the ground.


Golf - Sierra

Golf-Sierra is offline  
Old 8th Apr 2019, 16:58
  #3640 (permalink)  
 
Join Date: Mar 2015
Location: Washington state
Posts: 209
The problem looks intractable to me but that is why "A team" engineers exist. I didn't think that they were going to get the tunneling machine that got stuck under Seattle working again, but they did. Whether or not the public ever trusts the plane and whether SouthWest and Boeing survive without a government bailout may be another question.

The overall problem is far more insidious than relying on a single sensor, and I think that is what they are running into. Moving the stabilizer seemed like an elegant solution but there are circumstances where moving it can get it stuck due to aerodynamic forces. This requires heroic efforts from the pilots to unstick it (whether or not that was a factor in either accident is being debated). Low cost airlines can't afford to hire hero pilots, and an alternative plane exists that does not require them (allegedly.)

What I don't understand is how the system met the requirement for a continuous pressure gradient on the stick (I'm probably phrasing that badly.) So you have constant pressure when pulling the jetliner up rapidly to avoid that drone in your path, which is great because you don't get a sudden light stick sensation that lets you pull up too far into stall, but now push down to get back to level flight. This is probably yet another stupid question, but with the stab trimmed down, aren't the stick forces pushing down going to be much lighter than they normally are? I haven't seen anything -- although I could have missed it -- that MCAS trims back up when the AOA decreases, that is supposed to be noticed and handled by the pilot. This may be a reasonable assumption, but remember this is the pilot who couldn't be trusted not to pull the plane into a stall!
Water pilot is offline  

Thread Tools
Search this Thread

Contact Us - Archive - Advertising - Cookie Policy - Privacy Statement - Terms of Service - Do Not Sell My Personal Information -

Copyright © 2018 MH Sub I, LLC dba Internet Brands. All rights reserved. Use of this site indicates your consent to the Terms of Use.