Ethiopian airliner down in Africa

[KRH270/12]

Originally Posted by infrequentflyer789

On that point - can you show where it is confirmed that they did have IAS disagree, because as far as I can see it isn't mentioned at all in the report.

IAS is divergent (expected as ADIRU does use AOA to correct it, and AOA is massively diverged), but IAS DISAGREE isn't confirmed. I am pretty certain it should have happened, particularly given that it did with LionAir with AOA far less divergent, however there are several oddities in the narrative and traces that I can't get my head round at all.

eh, the FDR readout from the preliminary report shows some 20kt disagree and the non normal checklist for AOA disagree mentions possible IAS and ALT disagree....

so i wonder whats going on in the ADIRUs... the MAX documentation I got is no help at all...

[gums]

Salute Water !

Great point about hybrid systems as we see with this plane. I do not normally like big quotes, but need to include to set the stage

Quote:

Originally Posted by
spornrad Is there any remote possibility that software reduces the thumb switch authority on the left, not right, in those conditions (AOA disagree/MCAS activation) ?
Water replies:

Not particularly remote since it was the left side that the shaker was active on. We don't really know how the thumb switches operate, at first I thought that they were just switches to a solenoid but it seems more likely that they are just inputs to the fancy two computer/four processor 'black box' that is part of the flight control system. At that point, anything is possible since it is software.

If the data we have thus far on the architecture is close to accurate, then the MCAS software or maybe a switch somewhere that feeds the software or ..... ignores some of the previous model features that disabled some trim functions if the control column was pulled or pushed in opposition to what HAL was commanding.

It's the problem with a hybrid system, and a kludge one at that. The 'bus and military FBW systems give up from the beginning, and admit that the computers read control inputs and then give the pilot all the capability the plane has. The interfaces with the controls and computers are strightforward and easy to troubleshoot. It's also easy to determine whether you have a sensor problem or a stick problem or faulty gear/flap position switch and so forth.

Hate to break the bad news to our old, experienced 737 pilots who were unaware of MCAS and how it was implemented, but this new kid on the block ain't the one you flew 5 years ago. We now have had three documented cases where all the pulleys, levers and cables were not enough to overcome stabilizer trim position due to a kludge hdwe/sfwe "fix" to meet the aerodynamic requirements of the certification requirements. You know? Make the thing act/feel like "real" planes are supposed to act/feel.

Since 1973, we have only had a half dozen or so FBW failures in the F-16 that resulted in loss of the aircraft. They were primarily related to power supply design problems, wiring harness chafing, and in one case the loss of the radome due to a pelican impact that destroyed AoA and other air data sensors ( plane flew for another 8 or 9 minutes using just rate and attitude sensors before the pilot used the nylon let down).

I would also put up the Airbus statistics along with those of that little jet I first flew 40 years ago and the newer ones like the Raptor and Stubby (F-35). And be advised that the Stubby does not even have hydraulic lines to some control surface actuators. They are powered by electric motors or electrically powered hydraulic pumps. Makes for survivability if one of your ailerons is blown off by flak, see?

Gums sends...

[jimtx]

Originally Posted by Lost in Saigon

It is pretty basic. An aircraft has to fly like an aircraft. If you pull the nose up, and then release back pressure, the nose must return to somewhere near the original attitude and speed. If you disable MCAS, the B737 MAX will not meet the FAA stability requirements of Sec. 25.173 (Static longitudinal stability)

I suspect that without MCAS there would need to be a major aerodynamic redesign to meet the stability requirements.

The autopilot does not need MCAS. I don’t know which part of 25.173 MCAS was there for but I thought it was the stick force gradient. There is an exception to the trim speed requirement if exceptional attention is not required of the pilot. The lack of concern of FAA/Boeing to identify an envelope of concern if you actually had a malfunction involving turning off MCAS makes me think you could certify the MAX without MCAS if waivers are allowed for Part 25.

[astrodog3]

Originally Posted by Lost in Saigon

It is pretty basic. An aircraft has to fly like an aircraft. If you pull the nose up, and then release back pressure, the nose must return to somewhere near the original attitude and speed.

All indications are that the MAX does this.

What MCAS does is insure that the pilot must keep pulling on the control column harder and harder to continue pitching the nose up further as AoA rises. It was added because the regulations do not allow the resisting force of the column to decrease as elevator input increases and AoA rises.

The Lion Air and Ethiopian crashes seem to have occurred because the aircraft was unable to accurately judge if it was in this flight regime, and in the short, distracting time available to them, it did not occur to the pilots to perpetually counter AND trim from MCAS with ANU trim of their own or to trim the aircraft to neutral and then disable electric trim completly.

[1066]

Airline Influence

If it is true that SWA were a major push for the common type rating for the Max, (comments somewhere above in this thread that SWA could be in line for $1m per aircraft ordered if sim time was required for Max conversion), SWA were consistent with the way they influenced the introduction of the NG again with the intention of a common type rating for the NG and the 'classic' 300/400. I understand that the NG could have had a flight deck much like the 747-400 but this was ruled out to ensure the common type rating.
I'm not blaming SWA for any part in these two accidents but I think their part, as a very large customer, in the development of the 737 is relevant.

1066

[TachyonID]

SWA Training Requirements

Originally Posted by 1066

If it is true that SWA were a major push for the common type rating for the Max, (comments somewhere above in this thread that SWA could be in line for $1m per aircraft ordered if sim time was required for Max conversion), SWA were consistent with the way they influenced the introduction of the NG again with the intention of a common type rating for the NG and the 'classic' 300/400. I understand that the NG could have had a flight deck much like the 747-400 but this was ruled out to ensure the common type rating.
I'm not blaming SWA for any part in these two accidents but I think their part, as a very large customer, in the development of the 737 is relevant.

1066

Well, one of the "lined up holes in the cheese" here will undoubtedly be the question of which parties provided fiscal incentives to cut corners.
As an aero engineer, I'm appalled at Boeing's response. It is after all their design.
As a pilot, I'm disappointed in the design decisions that led to this point.
As a stakeholder in the aviation industry, though, we'll eventually get to the question of whether the industry leaders with the purse strings applied too much pressure that rippled through some bad decision making. If SWA offered $1M per plane to "not need additional simulator time"... well it will get interesting.

[fergusd]

Originally Posted by RatherBeFlying

I spent a large part of my career deep in mainframe Assembler applications and operating systems. A lot of it was really good and some was absolutely dreadful.

The principal determinant of success was elegance (or lack thereof) in design.

Same applies to C, C++ and all the wonderful new development environments that these days are proliferating faster than I can keep count. You may as well be in the fashion industry as bleeding edge IT.

Likely the A & B folk are sticking to well understood and proven development environments that are behind the times.

Simplicity, understandability and testability . . . and these days provability.

Modern safety standards strictly limit the use of assembler, because it meets none of the criteria above at all.

C is going the same way on some of those criteria IMHO especially as software complexity inevitably increases, C++ with very strict limitations (basically like very strongly typed C) has many things going for it with appropriate mitigations.

I don't think anybody would consider what I describe as anything other than archaic in terms of modern software development, about as far away from fashionable as it's possible to get.

;-)

Fd

[TryingToLearn]

Originally Posted by .Scott

Having programmed in machine language, I would NOT recommend it. It would be very difficult to reach the level of confidence for direct machine code (or even assembly) that would be required for this software.

The sequence would be: requirements, requirements review, design, design review against the requirements, test development based on the design, test procedure review, coding, code review, code testing. This requires code that can be examined by several team members with no chance of misinterpretation.

I fully agree, as an example the automotive functional safety process has the following steps:
-> Hazard & Risk analysis
-> Functional safety concept
-> Technical system safety concept
-> System achitecture
-> Technical Software Safety concept
-> Software architecture
-> fine design
-> implementation (code writing)
-> Module test
-> SW integration test
-> System integration test
-> System test
-> Vehicle Integration test
It is recommended to write technical safety documents in formal language to exclude misinterpretation. Implementation is less than 10% of the work. Toolchain qualification is also an important part of the process. Even the best compiler may cause errors if the memory module within the programmer's laptop has defective bits... (Yes, it already happened).
All documents are to be reviewed, accessed, there are walkthrough meetings and so on. All requirements need to have verification criteria specified together with the requirement and test cases are later based on there criteria... Within accessments, certain levels of safety require a certain independence between accessor and author (other team, department, division, company...).

Safe code can be done and if this was skipped just because one feared a diagnosis (AoA disagree), reaction (deactivate MCAS) and pilot teaching (continue flying, you probably never need MCAS anyway), this is a violation of safety culture beyond my imagination.
Fun fact: Emission standards for cars (onboard diagnosis 2) require 2 out of 2 for every sensor which may cause the violation of emission standards (ULEV, EU6...) and the engine control light on disagree. Seems like this is more important than a few hundred airplane passengers...

Oh, just one question:
People claim that the manual trim may not be operable in certain flight conditions while the electric trim motor is more powerful.
On the other hand the manual states that in case CUTOUT does not work, one should grasp and hold the wheel (?against the motor?). Did I miss something?

[MarcK]

Originally Posted by fergusd

I don't think anybody would consider what I describe as anything other than archaic in terms of modern software development, about as far away from fashionable as it's possible to get. ;-) Fd

Maybe someone thought that "Agile" was the way to implement MCAS.

[sycamore]

Anyone care to say what the aircraft trim change is at 350kts if you pop the speedbrales...?

[FrequentSLF]

Originally Posted by Water pilot

Not particularly remote since it was the left side that the shaker was active on. We don't really know how the thumb switches operate, at first I thought that they were just switches to a solenoid but it seems more likely that they are just inputs to the fancy two computer/four processor 'black box' that is part of the flight control system. At that point, anything is possible since it is software.

I am not a pilot, but it happens I can understand electrical schematics, the stab one is messy, clearly has been patched and subject to additions over the years with a mix of relays logic, analog and digital. The thumb switches are inputs to a fancy block, but they are hardware interlocked by column swtiches, cutout, etc. MCAS does not even compute the cutout switches...i do not believe the SW has any major control on thumb switches, Imcannot post the circuit from mobile, but,was posted earlier on this thread

[VicMel]

Originally Posted by bsieker

As we now know, MCAS is just such a system. So, as others have pointed out repeatedly, at least in hindsight, anything less than Level A is not appropriate.

It did not need loss of an aircraft to decide MCAS should be Level A, any software that has direct control of a piece of equipment, which if erroneously controlled could lead to a fatality, IMO must be Level A. So, MCAS should have been Level A from its original conception, if nothing else, in order to reduce the risk that the software will get “stuck in a loop” and just drive the stabiliser straight to its physical limit as fast as it will go! And I’m not too keen on the idea of putting non-Level A software in the same box as Level A software (assuming that the rest of the FCC software is Level A). Proving isolation and non-interference is very difficult.

Originally Posted by bsieker

Objectives that need to be demonstrated include things like

• High-level requirements are accurate and consistent.
• Low-level requirements are verifiable.
• Software architecture is verifiable.
• Source Code is verifiable.
• Source Code is traceable to low-level requirements.
• Source Code is accurate and consistent.
• High-level requirements are accurate and consistent.
• Low-level requirements are traceable to high-level requirements.

And many more. That is "the cheapest fix possible". It's not cheap, but it's doable in significantly less than 8 years for a company which has the procedures in place, which Boeing does.

Bernd

Boeing must realise that if there is another software “event” involving a 737 MAX, even if it has nothing to do the AoA, they will lose the public’s confidence completely. They really need to do an over-the-top job here, MCAS (and possibly other systems) need to be upgraded to A+. This should include (at least):-
a) fully independent (i.e. not another department of Boeing) verification of the items in the “objectives” list.
b) verification of robustness of code, preferably using tools to check for such things as poorly defined end condition of loops, depth of subroutine calling to prevent stack corruption, full timing analysis to prove there is no “timing overloaded” path etc., etc.
c) asking “what ifs” involving pilots, hardware team and experienced assessors, e.g. what information would pilots need to know to deal with a fault, what hardware idiosyncrasies need to be considered, what else could go wrong.
d) ensuring that every input is either from another Level A system, or validated by the new MCAS software
e) storing important status (e.g. flaps up) as a code and its complement, not as a single bit.
f) making MCAS an integral part of FCC i.e runs in both channels, if the outputs disagree the controller (could be the pilot, which would be a bit radical!) decides what to do
g) upgrading other systems to make them more robust e.g. ADIRU.

Redoing MCAS as Level A ready for submitting to the certification process should be relatively straight forward, I'd estimate 1 to 2 years; how long to ensure that supporting systems are brought up to an appropriate level (ADIRU to Level A?), 2 to 4 years perhaps. Achieving global certification, no idea.

[threemiles]

Originally Posted by yanrair

What was the most accurate speed readout on the plane indicating, during AF447 and the recent two MAX incidents - that being the GPS?? In AF447 it was reading something like 450 kts at the time of losing IAS. It is not going to change unless you do something like change pitch or power. Which is what happened to AF447 of course. You can fly on GPS speed for a long time until you have sorted out the problem. You can fly an immaculate circuit to land using just GPS. Yet in all the posts so far I have not seen much reference to its use in sorting out conflicting IAS/Stick shaker style events. Climbing out at 15deg pitch, 200 kts IAS Full power. GPS will be reading something similar, depending on wind and altitude. All hell breaks lose ( I am ignoring MCAS here which is a separate matter). Indicated speed all over the place. IAS disagree messages. Stick Shaker going off (one side failure) - what to believe?? Your GPS PITCH AND POWER. They are real, they are going to work and are unaffected by the ADIRU which relies among other things such as AOA and Indicated Airspeed.

There is a THIRD and independant IAS speed tape on the flight deck, right above the gear announciators.

​​​​GPS delivers ground speed and is of little value, unless you are close to MSL in calm air.

[737 Driver]

Originally Posted by sycamore

Anyone care to say what the aircraft trim change is at 350kts if you pop the speedbrales...?

That maneuver is above Vmo, so unlikely anyone here could say. However, just below Vmo I seem to recall that there is a very mild pitch up and completely manageable.

[njc]

Originally Posted by Derfred

Furthermore, it is apparent that MCAS is unaware of the position of the stab cutout switches, because it still tried to trim while the switches were in cutout.

Well yeah, it tried to trim, but only once. Does that mean that it figured out it was having no effect or that it only activates once and then waits until the pilots hit the trim switches again before having another go?
I can't understand how the latter would make any sense or indeed how it could have satisfied the requirement for which MCAS was invented. The former doesn't make a lot of sense either though.

[ams6110]

Originally Posted by TryingToLearn

I fully agree, as an example the automotive functional safety process has the following steps:
-> Hazard & Risk analysis
-> Functional safety concept
-> Technical system safety concept
-> System achitecture
-> Technical Software Safety concept
-> Software architecture
-> fine design
-> implementation (code writing)
-> Module test
-> SW integration test
-> System integration test
-> System test
-> Vehicle Integration test
It is recommended to write technical safety documents in formal language to exclude misinterpretation. Implementation is less than 10% of the work. Toolchain qualification is also an important part of the process. Even the best compiler may cause errors if the memory module within the programmer's laptop has defective bits... (Yes, it already happened).
All documents are to be reviewed, accessed, there are walkthrough meetings and so on. All requirements need to have verification criteria specified together with the requirement and test cases are later based on there criteria... Within accessments, certain levels of safety require a certain independence between accessor and author (other team, department, division, company...).

Safe code can be done and if this was skipped just because one feared a diagnosis (AoA disagree), reaction (deactivate MCAS) and pilot teaching (continue flying, you probably never need MCAS anyway), this is a violation of safety culture beyond my imagination.
Fun fact: Emission standards for cars (onboard diagnosis 2) require 2 out of 2 for every sensor which may cause the violation of emission standards (ULEV, EU6...) and the engine control light on disagree. Seems like this is more important than a few hundred airplane passengers...

And yet Tesla, an automotive company which presumably follows this process, still has an "auto pilot" software function that on more than one occasion drove a car into a stationary object at 70mph.

I would have little doubt that the software people at Boeing know how to develop software for any level of assurance needed. The question is why was MCAS not seen as a "critical" system?

[A. Muse]

I have followed this thread from the beginning, and read all of the posts, and some of the deleted ones, (thanks to the mods for keeping things on track and removing the abuse).

Two posts stick in my mind and I have been waiting for someone with more knowledge than me to put 2 and 2 together and make 22, so I’m ready to be shot down in flames.

Way back someone posted that in the rush to market Boeing might not have given full instructions for the installation of the wiring in the Max as was normal practice with previous builds. IF this is a fact and not hearsay or fabrication, is it possible that a wrongly routed, worn, stretched or chafed wire or faulty connection is partly to blame for the AoA readings? I link this to Post 3234 by jimjim1

If the vane had been lost the AoA sensor would become unbalanced about its usual axis of rotation. The internal balance weight** would then cause the axle to be subject to movement when the aircraft transitioned from +g to -g. +g would cause the indication of +AoA. (If I have got this the right way round).

Looking at the FDR traces it can be seen that this appears to be the case. I have drawn four green vertical lines to indicate the transitions from +g to -g and vice versa. In each case they appear to align with a change in the direction of movement of the sensor in the correct sense. Remember that the data consists of discrete samples and we do not know the sample rate and I am assuming that any small discrepancies are due to errors introduced by the sampling.

I have (rather crudely) chopped out a period in the middle of the chart so that it is a bit narrower so that the scale markings can be easily seen. The horizontal blue line in the "g" section of the chart is coincidentally exactly on 0g.

It therefore seems quite likely that the vane was lost or perhaps damaged soon after take off, perhaps by a bird strike or otherwise. Note however that if the vane had been bent back its balance would be moved in the other direction and its aerodynamic influences would still have been felt so I think that the best conclusion consistent with the data is that the vane was lost.

If the transition from +g to -g caused a wiring loom or connector to move and interrupt a current or produce an unwanted one, would this have the same effect on the instruments and controls as a faulty AoA vane?

Sadly we can never know how things were installed on the two lost aircraft, but checking of the routing and condition of cables and connectors in the grounded planes might be advisable.

[ecto1]

Electrical gremlin highly unlikely in Ethiopian event, because the vane readings when the plane dives are there, and spread all over the range. The vane essentially turned into a g-meter, either by loosing the exterior part of it or the connection to the exterior part of it.

Indonesia, OTOH, it is possible, although highly unlikely. However, It is still in my eyes the most probable explanation (short signal to ground with high resistance).

[threemiles]

Originally Posted by weemonkey

How is it sourced though?

Aux pitot & alternate static sources, no ADC/ADIRU’s.

[infrequentflyer789]

Originally Posted by weemonkey

How is it sourced though?

Pretty sure it is from the third set of pitot/static sensors, without AOA correction - since there isn't a third AOA sensor.