Ethiopian interim report on Max 737

[porch monkey]

The format and the intention of indents and other matters are stated in the QRH document. Hence, the danger of quoting a checklist in isolation. Boeing, and Airbus for that matter, go to some depth regarding how and when the different checklists are used. The fact that you don't fly becomes obvious.

[Flightlevel41]

From page 15 of the report:
At 5:40:00, the first MCAS nose-down trim occurs for 9s. This was after the autopilot had disconnected after 32s of engagement on the crew's third attempt at engaging the autopilot (was this an appropriate action at this point in time?).
After the MCAS activation, the PF needed 90lbs of force to keep the aircraft from pitching nose down.
At 5:40:14, there was manual electric trim for 2s which brought back the stabiliser from 2.1 units to 2.3 units in a nose up direction.
At 5:40:22, the second MCAS activation took place for 7s, which was interrupted by both pilots applying nose up electric trim for 9s.
So in total, we have 18s of automatic nose down trim, but only 11s of nose up trim in opposition.
At the end of this phase, the stabiliser was at 2.3 units with the crew needing 94lbs of force to keep the jet at a 2.5deg pitch up angle.

Some questions here are:
Why were the crew not using the electric trim more aggresively not only to counter MCAS but also to offload the large amount of pressure needed to hold the control column in a nose up attitude? Because they used electric trim they must have known this wasn't a "true" trim runaway situation and so still had control over the horizontal stabiliser.
Did they rush to cut-out the trim without neutralising the stabiliser first?
Also, did the crew have to answer every ATC call and follow heading instructions with a sick airplane? Could they not have asked to standby in order to reduce potential distractions?

I'm a non-pilot just looking to get some additional perspective here. Perhaps there is some context that has been omitted from the CVR or FDR which would explain the pilot's behaviour. To me the information provided so far makes it seem there were opportunities to save the aircraft that MAY have been missed (again need to wait till the final report to make sure).
None of this is take away from a poor Boeing MCAS design or inappropriate FAA oversight. However, I think too many people make the assumption that a bad MCAS design simply means the pilot's have no means to save the aircraft once uncommanded trim begins.

[FullWings]

I agree with alf that you have to take a holistic approach to see why these accidents happened. If you look for “holes in the cheese”, they appear right from the start in a progression through design, testing, certification, documentation, training and instruction. The MAX congressional report backs much of this up.

There is a world of difference between a continuous trim runaway at altitude and level flight in the sim, knowing it’s going to happen, and something that presents without warning as UAS and/or stall combined with intermittent trim inputs at a phase of flight where intermittent trim inputs are expected and the ground is not far away. Add sensory overload to a complicated decision tree to navigate and it is no wonder that things went badly. Boeing internally estimated that you had <10s to correct the problem before it became possibly unrecoverable - this was before any of the crashes happened.

Yes, it could have been handled better but this was a multiple failure scenario at a critical stage of flight; we also don’t know if the MAX or other variants can be manually trimmed at Vmo with the stab a long way out-of-trim forward (the roller coaster doesn’t work at low altitude).

[ST Dog]

Originally Posted by DaveReidUK

If the two versions of the checklist are "sort of the same", why (in your view) was it considered necessary to make the amendment at all ?

I didn't say the checklists were the same. Just the end results of the procedures.


As for why change, well clarifications are made all the time. What made sense 20yrs ago when writing something may not make sense today due to changes in language and shared knowledge (assumptions).

[DaveReidUK]

Originally Posted by Tobin

Now, I can stand to be corrected. Is the format of the checklist officially described anywhere? How are the indented statements to be read? Are they simply independent instructions to be executed always?

They don't look like it to me. They each look like a dependency or continuation of the numbered step immediately above. My natural reading is to get through "2. Autopilot (if engaged) ....." and think Okay, autopilot is definitely disengaged. The rest of this is irrelevant. What's step 3? Then move onto step 3.

Correct.

As alluded to above, a QRH typically contains an explanation of the conventions used, symbology, etc (for example, what a row of square dots means), which can run to several pages.

But it would appear that Boeing (and presumably Airbus) doesn't consider it necessary to spell out that a checklist item of the form "If (condition X exists) perform (action Y)" means that whether you do or don't do Y depends on X. Presumably that is considered sufficiently obvious as to need no further explanation.

[Snyggapa]

I don't fly aircraft but I do design systems and care a lot about the "human interface" part. Things I design won't kill anyone but will cost an awful lot of money in a split second if used incorrectly - and as such I spend a lot of my time watching and listening to people using what I design in real world situations, and forensically taking apart what happened when something goes wrong. There are a number of things that have become clear over time.

Human ability to misunderstand your design intentions or to think that the system does more than it really does - is almost unlimited.
Human ability to think that they know: that because this does X then if I use it this way it must do Y - is almost unlimited.
A designer's ability to comprehend all possible permutations of X and Y is limited. Sometimes users view of X and Y are simply illogical, but often they have a point. Sometimes what you design may be perfect in theory but just simply not the way that the world works in practice.
Human ability to quickly read a message or react to something unexpected when they are preconditioned to react in a different way is very poor.
Particularly so when they have done the same process 100 times before, seen the same error and ignored the same error 100 times , then on the 101st go you display a different error that is important. The critical stuff gets lost in the noise.
Once things start to go off-piste and the pressure is on, some individuals stop, think and analyse. For others, everything goes to rats, logical thinking goes out of the window.
Language barrier is a problem. Everything that I do is in English but a fair proportion of my users are European so not in their mother tongue. Not only do they have to mentally translate back and forth, there are cultural differences too.

Out of curiosity, are the memory items always in English? Is there any possibility of a misinterpretation between the native English and the mother tongue of whoever is actioning it - either through language or cultural issues? Could some of the subtlety that is important and "obvious" to a native speaker be lost?

[Magnis]

Originally Posted by FullWings

...we also don’t know if the MAX or other variants can be manually trimmed at Vmo with the stab a long way out-of-trim forward (the roller coaster doesn’t work at low altitude).

The interim report conducted some tests

1.16.1 SIMULATOR ASSESSMENT OF CONTROL COLUMN AND TRIM WHEEL FORCE (pages 77-80) copied, paraphrased, my emphasis

3 simulation tests were conducted ... in a CAE manufactured B737 MAX level D full flight simulator

Speed 220 Hands off trim value 6.8
Mis-trim value 2.3, Assessment value D
Mis-trim value 3.3, Assessment value C
Mis-trim value 4.3, Assessment value B

Speed 250 Hands off trim value 6.2
Mis-trim value 1.7
Mis-trim value 2.7
Mis-trim value 3.7 , Assessment value A

Speed 300 Hands off trim value 5.3
Mis-trim value 0.8
Mis-trim value 1.8
Mis-trim value 2.8, Assessment value A

A = trim wheel not movable
B = trim wheel barely movable (1 turn not completed)
C = trim wheel moves with great difficulty (2 turns not completed)
D = trim wheel moves with some difficulty (2 turns completed)

It was observed that the greater the mis-trim value, the greater the force required by the pilot
on the control column to fly level flight and consequently the greater the force required turning
the manual trim wheel.
As the trim value from the event flight was around 2.5 units by the time the crew tried to use
the manual trim wheel, even at a speed of 220 kts the difficulty level of turning the manual trim
wheel was level B (barely movable/ 1 turn not completed).


1.16.2 ENGINEERING SIMULATOR AND FLIGHT CONTROL TEST RIG ASSESMENTS (pages 80-82) - verbatim
(...)

TRIM WHEEL EVALUATION AT THE FLIGHT CONTROLS TEST RIG (FCTR)
Multiple scenarios were executed to run different manual trim Wheel forces for ET-302 flight conditions on ground as well as at different speeds and altitudes using Flight Controls Test Rig (FCTR).

A trim wheel evaluation was performed at the Flight Controls Test Rig (FCTR). Tests were done with aircraft on ground as well as at different speeds and altitudes with different trim settings.
It should be noted that:
- The maximum possible mistrim setting on the FCTR is -1.5 unit.
- 15 wheel rotations are necessary for 1 unit of trim.
- The first test was conducted with aircraft on ground at 0 kt. Expected force on the wheel 10 Lbs.

• It was noted that the wheel was easy to operate, the FCTR matched the physical aircraft very closely and that it was qualitatively close to CAE training simulator forces. The FCTR instrumentation recorded static force of approximately 9.3 pounds.

The second test was conducted with aircraft at 12,000 ft, 250 kts, in trim condition (expected 15 lbs force).

• It was noted that the wheel was a bit more difficult to operate but that it was still doable with one hand. It was qualitatively close to CAE training simulator forces. The FCTR instrumentation recorded static force of approximately 15 pounds.

The third test was performed at 12,000 ft, 340 kts (VMO), in trim condition (expected 21 lbs force.

• It was noted that the trim wheel force become much more difficult to operate than in condition 2. The wheel motion became jerky, straining efforts to turn it. Impact on speech. Qualitatively more difficult than on CAE training simulator. 15 turns would be tiring. Rig instrumentation recorded static force of 21 pounds.

The fourth test was conducted at 15,000 ft and 340kts (VMO), -1.5 units (mist-rim) 14 , expected 35 lbs force.

• It was noted that it was impossible to turn the wheel with one hand confirming the first officer’s statement “it was not working” meaning “hard to move. Some participants expressed surprise at the difficulty. It was possible to turn the wheel with two hands although not convenient at all. The level of force for this condition was found to be between 30 and 40 lbs. It was agreed that difficulty would increase further outside the normal operating envelope (as in the accident case).

The mistrim level on the event flight was 2.5 units at 340 Kts but since the FCTR is limited to 1.5 unit of miss trim, the actual event flight condition could not be tested. Manual trim forces were not attempted in the ECAB as it was not calibrated for the MAX. Manual trim was evaluated in the Flight Controls Test Rig (FCTR).

[KeepItStraight]

For those non flyers reading this thread. The QRH is a document that any crew should be very well versed in. The trim runaway scenario, for example, that has been mentioned in recent posts should have been practiced in training for the type rating and would be covered during simulator checks. The MCAS events will not have been the first time these pilots saw these procedures.

It's not like picking up the trouble shooting instructions for some gadget at home and having the first read of them when it stops working.

Pilots no matter their mother tongue should be familiar with the correct procedure and any ambiguities that may exist should have been thrashed out during training.

[FullWings]

That’s all well and good but memory/recall items in a QRH have specific triggers, like engine fire -> engine fire checklist. Before the first accident, the trigger for trim runaway was something like “continuous uncommanded trim movement”, otherwise you’d be disconnecting it every time you took off (STS, MCAS, etc.) After Lion Air, there was realisation that it was more complex and you had a bit of diagnosis to do, e.g. is it normal operation? Will manual trim be possible if the electric is disconnected? What’s actually gone wrong? And so on. These checklists have to be simple because they need to be executed correctly in time-critical events and if there is more than a little bit of if-then-else, then the chances of success diminish rapidly.

Add in the symptoms of UAS, stick shaker, aural warnings, etc. and you have a recipe for confusion and overload. Would a scenario like this be covered in training/conversion? Unlikely as a) most courses are cut down to the bare minimum for cost saving, b) this is a multiple-failure event and, more importantly, c) Boeing sold the MAX on the basis no physical training was needed coming from another variant. Even the manuals didn’t have MCAS in them! There is also doubt as to whether commercial simulators could actually reproduce these effects at the time.

The available documentation evolved between the first and second crash and became somewhat confusing, IMO partly because it was written by lawyers rather than pilots. There were facts distributed across various notices and amendments, such as the effect of autopilot and flaps, which likely contributed to the overall level of confusion/indecision in the second accident. I think you also have to include the environment, which was high altitude, low level and terrain-constrained, so an awful lot to think about in a short space of time...

[safetypee]

FullWings, helpful inputs.

A core issue re MCAS it that no specific checklist was published; Boeing assumed that the existing trim-runaway drill would be used, which assumed situation recognition (*1 page 125).
The tests and evaluations identified the criticality of timely response - diagnose and act with 10 sec; yet the first 3-4 sec were 'certification-inactivity', and then 1 sec later (5 sec elapsed) the trim movement by design stopped. MCAS was reset; the human similarly 'reset' because trim movement was normal. There was no trigger feature to assess the situation as a failure.

The congressional findings are more damming;-
'In 2012, Boeing developed initial concepts for an MCAS annunciator to inform pilots when the MCAS failed, but never implemented it.' (*2 page 8).

and from page 10,
'Boeing’s own analysis showed that if pilots took more than 10 seconds to identify and respond to a “stabilizer runaway” condition caused by uncommanded MCAS activation the result could be catastrophic. The Committee has found no evidence that Boeing shared this information with the FAA, customers, or 737 MAX pilots'.

'The 10-second reaction time and the potential for it to result in catastrophic consequences was discovered early on in the development of the 737 MAX program.'

* 1 http://s3.documentcloud.org/document...rch-9-2020.pdf

* 2 https://transportation.house.gov/imo...rch%202020.pdf

[BDAttitude]

So pilots were supposed to stop a runaway within 3s, when they first have to identify continuous movement, then execute 4 items of a check list of which the last one is again to identify continued continous movement before hitting the cut-off switches. And as professional pilots you are exculpating this? Interesting.

[safetypee]

No, not 'exculpating' - show or declare that (someone) is not guilty of wrongdoing; and neither the converse.

The purpose of establishing the processes involved in certification, the limitations of test systems, assumptions made, commercial pressure, etc, is to better understand normal human performance within behaviour shaping situations, and to do that without hindsight bias.
Similarly for the accident crews, establish the detail of the evolving situation which they had to identify and manage from after takeoff to the end of flight.

And from the above identify any mismatch which would require a change of action - design, documentation, training. A difficulty is that whilst both the manufacturer and regulator work in the real time - 'now', a crew operating the aircraft system under review will be in some unspecified 'future'.

In order to compare these the reviewers require 'Requisite Imagination' - The fine art of anticipating what might go wrong. (*)
The further reference involving culture is ironic in that the congressional report subtitle chooses 'cost' before 'lessons' (safety).
Also note the role of language in culture; even nations using English can adapt words to mean whatever is required for them to mean, particularly change of meaning with situation. (e.g. TCAS 'advisory')

https://www.researchgate.net/publica...might_go_wrong

also

Cultures with Requisite Imagination, https://link.springer.com/chapter/10...662-02933-6_25

https://www.researchgate.net/publica...avioral_Issues

[nicknamepilot]

Originally Posted by alf5071h

These views rely on assumption, that a crew would assess the situation as trim runaway and choose that checklist - and execute it correctly. Pprune has discussed the piloting view extensively, and further debate is unlikely to changed 'fixed' minds

that’s wrong. Whether or not the trim runs and then stops, it doesn’t matter. What else are you going to do about it? Just let it do that for the entire flight until you’re 20 feet in the ground? I’ve had runaway trims that stopped and went again, some slow others fast. A wire making intermittent contact with a ground will do just that.

While Boeing’s practices have definitely come into question, so is the pilot performance as this crew did nothing about it. All the captain did was try to engage the autopilot, 6 times. They ran the correct checklist. And only completed one step on it while ignoring everything else. The F/O manually trimmed in the wrong direction and then finally reversed the only step completed and trimmed up from 2.1 to 2.3. I don’t fault them for reversing the step, anyone would, but I wouldn’t if I was going to trim that much. I’ve always been under the impression that any crew in this situation would reverse the step to actually trim the **** out of the plane. And then put the switches back afterwards, which they didn’t and had 10 seconds to do it. When they specifically confirmed the emergency; that the problem was with the trim. They knew it was running, and still disregarded the switches after trimming. Pilot performance was as bad as it gets, and call’s Ethiopia’s training into question. They obviously don’t teach their pilots anything about the trim. They likely tell them to just let autopilot handle it

[A37575]

Pilot performance was as bad as it gets, and call’s Ethiopia’s training into question. They obviously don’t teach their pilots anything about the trim

With the advent of the Boeing 737 Classics, Boeing withdrew its advice published in the original Boeing 737-200 Pilot Training Manual (PTM) on use of the manual stabilizer trim wheel to counter a runaway electrical stab trim motor. This was the so called "Roller-Coaster" or "Yo- Yo method of recovering from an extreme stabiliser position. Previously this method was taught in the Boeing 707 for the same reason. In fact it was even used for airborne demonstrations with certain limitations.

The wording in future editions of the FCTM was amended to talk about "aerodynamically relieving" stabilizer forces. In other words the same meaning as roller coaster but more obtuse; especially to English second language operators. Even today, ask 737 flight crews the meaning of the term "Roller Coaster" as it applies to manual stabilizer trim operation, chances are they wouldn't have a clue what you are talking about.

In both the Boeing 737 MAX crashes it is a good bet the crews had never practiced this recovery in the simulator because their instructors wouldn't have had a clue on past history either. You can leave it to your imagination where the blame lies for that critical omission.

[alf5071h]

nicknamepilot,

Your conclusions about piloting performance and training suggest bias; without any supporting evidence of what was trained, what the crew knew, what was recalled or not.
Furthermore, the narrow viewpoint fails to consider the overall context of the accident and preceding events of the flight.
The crew were faced with a surprising situation after lift off with several simultaneous system alerts.
Stick shake might have been priority; - having established safe flight, then considered airspeed disagree which might suggest unreliable airspeed.
It is assumed that crew had sufficient understanding of the situation for adequate control at that time.

Only when the flaps were retracted did the trim abnormality - MCAS failure, appear; without alerting, or prior knowledge of an unknown system.
Trim change is cued to pilots as a change in stick force, trim is used to counter this, as would be done for flap retraction. Stick force also changes with airspeed, and even thought the crew used trim, the aircraft did not respond as expected. If (a further assumption), the previous, strong understanding of unreliable airspeed dominated awareness then the unexpected trim changes could have been related to speed error. There may have been doubt that the aircraft was flying at the correct speed, thus maintain thrust, etc, and thereafter the situation deteriorated, the crew struggling to update previous understandings to deduce a failure of an unrelated system which no one knew about. Thence …

Blame is a useful point of closure for weak understanding of human processes. We dislike the unknown uncertainty and often insert items situational factors only known with hindsight, into a hypothetical situation which we form to provide closure; i.e. why we wouldn't make similar errors in the same situation - yet we don't know.
Why then conclude that the crew knew.

[infrequentflyer789]

Originally Posted by alf5071h

Only when the flaps were retracted did the trim abnormality - MCAS failure, appear; without alerting, or prior knowledge of an unknown system.

Agree but for a couple of pedantic nitpicks: On ET I am pretty sure MCAS only appeared when autopilot disengaged (it was engaged for half a minute), which I think was after flaps up. Also, MCAS wasn't an unknown system at that point since it had been revealed in LionAir crash investigation, in fact if you look at e.g. ACN: 1597380 it's clear that some pilots at the time (after LionAir crash) both knew about MCAS and briefed preventative actions against it. Unfortunately the preventative action was "engage AP".

Additionally the first pitch-down (or at least pitch reduction) for ET was under autopilot and therefore in theory not MCAS. It's not clear why AP got confused, dodgy AOA at a guess (but I don't think that has been confirmed anywhere), nor is it clear why it disengaged (although GPWS alert sounded shortly after...). There have been other reports of MAX AP unexplained pitching down shortly after takeoff (one is the ACN above), to my knowledge this (may not all be same issue) remains unexplained. Obviously the solution is disengage automatics and pull up, but then MCAS comes into play...