Re the NAS report - post #186
“This report responds to the statement of task specified in the Aircraft Certification, Safety, and Accountability Act. It must be noted at the outset that this report does not assess the application of the TARAM process to any specific incidents or accidents, including the 737 MAX accidents. While the committee was provided a copy of the 737 MAX TARAM analysis provided by the FAA to Congress in late 2019, FAA management declined to provide additional details or to discuss the TARAM analysis of the 737 MAX with the committee. The committee, therefore, was unable to comment on the 737 MAX TARAM analysis. Regardless, the committee was able to make recommendations that, if adopted, would significantly improve the TARAM process.” ( my emphasis )
However, the recommendations to the FAA to review their process addresses several issues which are suspected to have been factors in the 737 Max certification; i.e. - recommendations:
Designate new FAA expertise; difficult if this was lacking, no overnight experts.
Define the type of data to be monitored; ‘definitions’ tend to restrict, and ‘monitor’ implies after the fact, not aiding initial type certification.
A significant issue is to quantify human performance. The Max certification assumed (or was persuaded to assume) that the pilot would be capable of managing an abnormal situation (without guidance and training, etc).
Conceptually this is a continuing issue in all certifications which require expert judgement (normally manufacturer); it challenges the ideas of inservice HF expertise, and if the required performance (the judged level of safety) can ever be quantified.
The report continues with recommendations on computation and analysis, implying that human attributes and uncertainties in operation can be expressed numerically.
Probabilistic risk assessment is increasingly difficult in safe industries because of reducing safety data, thus depending more on subjective assessments; computers lack subjectivity (other than that given by humans)
And so on …; tasks and challenges which the FAA, or any other authority might not achieve.
The meaningful issues involve the need to look at safety differently, not rewrite the book (it has got us this far), but to use alternative views as additions and enhancements; considering the human as an asset, error is normal, considering the wider system, etc.
This could become an industrywide issue particularly if other regulators use the NAS report or revised FAA guidance because of the salience of the 737 Max failure, opposed to thinking much deeper, wider, about the uncertainties in modern operations involving human activity at all levels; Congress, NAS, FAA, where individuals’ personal qualities might only be a small part of the need to think about safety differently.