An international panel of air-safety regulator experts convened by the Federal Aviation Administration released a damning report Friday that criticizes both Boeing and the FAA for the way they assessed and approved the design of the 737 MAX automated flight control system implicated in two fatal airliner crashes.
More broadly, the panel’s review questions how systems on the the MAX were certified as derivative of a now-50-year-old aircraft design and recommends the FAA examine the criteria for determining when an airplane is so different from the original model that it requires an entirely new type certificate.
The panel further recommended that airplane safety systems address the new reality of increased cockpit automation by building in protections by design and reducing the reliance on pilot action to respond to emergencies.
The Joint Authorities Technical Review (JATR) panel found that the MAX’s new flight control system, which played a central role in the accidents in Indonesia and Ethiopia that killed 346 people, “was not evaluated as a complete and integrated function in the certification documents that (Boeing) submitted to the FAA.”
As first reported by the Seattle Times on March 17, the panel found that Boeing submitted to the FAA for evaluation an inadequate technical description of the airplane’s new Maneuvering Characteristics Augmentation System (MCAS) that lacked full details of when the system activated and the extent of its power to push an airplane nose down.
The report states that technical details of MCAS were “not updated during the certification program to reflect the changes to this function within the flight control system.”
“In addition, the design assumptions were not adequately reviewed, updated, or validated; possible flight deck effects were not evaluated; the (System Safety Assessment) SSA and functional hazard assessment (FHA) were not consistently updated; and potential crew workload effects resulting from MCAS design changes were not identified.”
“The lack of a unified top-down development and evaluation of the system function and its safety analyses, combined with the extensive and fragmented documentation, made it difficult to assess whether compliance (with safety regulations) was fully demonstrated,” the report states.
Undue pressure
The report also found that the FAA had “limited involvement” in the evaluation of MCAS and left most of the work of assessing the system to Boeing itself.
“In the B737 MAX program, the FAA had inadequate awareness of the MCAS function which, coupled with limited involvement, resulted in an inability of the FAA to provide an independent assessment of the adequacy of the Boeing-proposed certification activities associated with MCAS.”
“In the context of the B737 MAX, the JATR team’s assessment is that MCAS should have been considered a novelty (and therefore clearly highlighted to the FAA technical staff) owing to the important differences in function and implementation it has on the B737 MAX compared with the previous MCAS installed on the B767-C2 (tanker).”
The report, confirming a Seattle Times report on May 5, also cites indications that Boeing employees working on the certification of the airplane on behalf of the FAA faced “undue pressure” from managers who prioritized cost and schedule.
“Signs were reported of undue pressures on Boeing … engineering unit members performing certification activities on the B737 MAX program,” the report states. It attributes the undue pressure within Boeing to “conflicting priorities and an environment that does not support FAA requirements.”
The report recommends revision of the system whereby the FAA delegates much of the oversight of airplane certification to Boeing, a system known as Organization Designation Authorization (ODA) within which Boeing appoints its own engineers to do the certification analysis and testing and they report to managers within the Boeing organization who relay the results to the FAA.
Sign up for Evening Brief
Delivered weeknights, this email newsletter gives you a quick recap of the day's top stories and need-to-know news, as well as intriguing photos and topics to spark conversation as you wind down from your day.
The JATR recommends adjusting this structure so that authorized engineers at Boeing be provided “open lines of communication to FAA certification engineers without fear of punitive action or process violation” to ensure they “are working without any undue pressure when they are making decisions on behalf of the FAA.”
This recommendation mirrors the advice of experts cited in that May Seattle Times story who advised that the FAA revert to elements of an earlier oversight structure — called “Designated Engineering Representatives” or DERs — in which the Boeing engineers who act on behalf of the FAA were appointed by the FAA and reported directly to their technical counterparts at the FAA.
Pilots unable to cope
The panel also addresses the assumption in the FAA regulations that pilots will recognize something wrong within a second during manual flight and will respond with corrective action within 3 seconds. The report indicates that the 737’s crew alerting systems that tell pilots when something goes wrong may not be adequate for such an assumption.
“The 3-second reaction time may not be appropriate, depending on the cockpit alerting philosophy and trim system architecture and controls,” the report states.
The JATR recommends that, when a system fault or inappropriate operation results in cascading failures and multiple alarms the FAA should address “how adequately the certification process considers the impact of multiple alarms, along with possible startle effect, on the ability of pilots to respond appropriately.”
“Inherent in this issue is the adequacy of training to help pilots be able to respond effectively to failures that they may never have encountered before, not even in training,” the report states.
Last week, the Seattle Times reported that Boeing pushed the FAA to relax certification requirements for crew alerts on the 737 MAX. In doing so, Boeing used a process called the Changed Product Rule.
In reviewing how this rule was applied to certification of the MAX’s flight control system, “the (JATR) team determined that the process did not adequately address cumulative effects, system integration, and human factors issues.”
JATR therefore recommends a top-down reassessment of how derivative models are certified, to determine “when core attributes of an existing transport category aircraft design make it incapable of supporting the safety advancements introduced by the latest regulations and should drive a design change or a need for a new type certificate.”
The report states that the FAA raised concerns to Boeing about the cumulative effect of cockpit system changes from the previous 737 model to the MAX that might create a need for simulator level pilot training.
“Boeing’s response to this concern was that there was no precedent” in previous certifications of derivative models. “The FAA accepted Boeing’s response on 26 January 2016,” the report states.
In a teleconference call Friday, JATR chairman Christopher Hart, former chair of the National Transportation Safety Board (NTSB)), said that the increasing prevalence of automation on aircraft means that “this is not just an airplane problem, but an airplane/pilot problem,” which he said complicates decisions about grounding and ungrounding an aircraft and is likely to become a major issue in future.
“As automation becomes more and more complex, pilots are less likely to fully understand it and more likely to have problems and more likely to encounter scenarios in real operations that they haven’t seen even in a simulator,” he said.
Hart called on the FAA and regulators worldwide to recognize and address “this new reality of super-complex automation and pilots not necessarily understanding how to operate it.”
The JATR report says that “as systems become more complex, the certification process should ensure that aircraft incorporate fail-safe design principles.”
“These principles prioritize the elimination or mitigation of hazards through design, minimizing reliance on pilot action as primary means of risk mitigation,” the report goes on.
The panel separately recommends that “the FAA should review the natural (bare airframe) stalling characteristics of the B737 MAX to determine if unsafe characteristics exist.”
This suggests JATR wants the FAA to assess the safety of the MAX without MCAS in operation. Boeing has said that the purpose of MCAS is not to prevent a stall but simply to make sure it handles exactly like the earlier model 737 when going through certain stall testing.
Some criticism of the company on social media has been skeptical of this, proclaiming the MAX “inherently unstable” because it needs software to fly safely.
To demonstrate otherwise, Boeing test pilots this summer repeatedly flew that required stall test on the MAX — an extreme maneuver called a “wind-up turn” — both with and without the revamped MCAS operating. Boeing says it is satisfied with the results.
The FAA and overseas regulators will conduct their own flight tests, likely next month.
MCAS upgraded
JATR was convened in April by the FAA to independently evaluate all aspects of the design and certification of MCAS. The panel is made up of technical safety experts from the FAA and NASA along with the civil aviation authorities of Australia, Brazil, Canada, China, the European Union, Japan, Indonesia, Singapore and the United Arab Emirates.
Boeing did not directly address the report’s findings Friday but said in a statement that it “is committed to working with the FAA in reviewing the recommendations and helping to continuously improve the process and approach used to validate and certify airplanes.”
FAA Administrator Steve Dickson issued a statement thanking JATR for its “unvarnished and independent” report.
“I will review every recommendation and take appropriate action,” Dickson said. “We welcome this scrutiny and are confident that our openness to these efforts will further bolster aviation safety worldwide.”
EASA, the European Union Aviation Safety Agency, called the report “thorough.”
“We will analyse all recommendations made to assess their relevance to the European system and take action wherever necessary,” EASA said in a statement.
MCAS consists of new flight control software added to the MAX. If a sensor that measures the jet’s angle of attack, the angle between the wing and the oncoming air flow, indicates that the nose of the aircraft is pitching up, MCAS is designed to swivel the jet’s horizontal tail — called the horizontal stabilizer — so as to push the nose of the aircraft back down.
The JATR report notes the failure in communication between Boeing and the FAA during the certification process as MCAS evolved “from a relatively benign system to a much more aggressive system.”
The result was a failure to address the potential unintended consequences that resulted from “designing software for one scenario — in this case, high-speed windup turns — and then modifying the software for a different scenario — in this case reducing the pitch-up tendency at higher angles of attack at low speeds.”
Boeing has prepared a redesign of MCAS that addresses the inadequacies of the original design, which was activated by a single angle-of-attack sensor. On both crash flights, the accidents were initiated by a false signal from that one sensor.
The updated MCAS software will be activated only if both such sensors on the aircraft show the same high angle of attack. In addition, the system is now redesigned so that it can activate only once.
And Boeing has changed the overall software system architecture to compare readings from both flight control computers, instead of using only one, and to shut down MCAS in less than a second if the computers disagree.
But as regulators evaluate those improvements and the pilot training that will be required, the 737 MAX remains grounded worldwide seven months after the second crash.
On Friday’s teleconference, FAA spokesman Lynn Lunsford said that because the FAA is doing an entirely new safety analysis of all the changes to the MAX before it give the plane clearance to return to service, “the majority of he return to flight issues that have been raised by JATR are being addressed.”
“We are going through the recommendations one more time to make sure that any of them that aren’t being addressed will be as part of the current review,” Lunsford added.
FWIW Lunsford covered Boeing and Aerospace issues for years at WSJ. He is/was an excellent reporter and knows many in the field.