During the congressional committee testimony given by the Chief Engineer, a fundamental flaw of the existing certification came out in a curious Q & A series, by both Democrat and Republican questions.
The engineer was asked whether the process that was alluded to as being the goal of the project worked and the answer was it had been effective, that process had worked. The fact that 346 people died was commented on by those conducting the questioning and resulted in about the only heated parts of the discussion. as one would imagine. The accidents were described as learning cases. They are, at some unfortunate cost in inconvenience to the N.O.K.
A system should learn from tragedies, to do otherwise is a disservice to the departed. However, the problem seems to this observer that the examination of the designs is done by the same people who do the design work, whether in the DER-NAA-ACO structure or ODA or DOA type systems, choose your poison. The viewpoint of the examiner is intimately entwined by circumstance to those doing the designs, and so the challenge of assumptions that a 4-year-old gives the designer of a toy doesn't occur. In HF terms, the potential for "group think" is high, and that effectively short circuits an examination of the merits of a product for compliance. Sure, anyone can tick boxes, as the testimony outlines, but there were no questions being raised, no curiosity, no what-ifs arising from the candor that comes from not having a common education, experience, assumptions, and biases as the person whose work is being evaluated.
Oddly, this is the same issue that arises from most QA audit processes, there is an emphasis on ticking checklists, and they get in the way of looking out and seeing curiosities in the design. Asking searching questions to understand how something works tend to get knocked out of the observer from the institution they end up working for.
Or give it to a marine, they know how to make things break.
e.g., 40 troops are stuck in 1st class in the back of a C-130; the loadmaster hands out 40 sets of EAR plugs, the foam yellow ones. After doing the rounds of the troops, the leader of the troops approaches the loadie and asks if he has more of the yellow candies, his guys had already consumed them all. How you think something works may be quite different to a 4 year old, or to someone who has a different point of view.
I would not say that it is the same people, exactly, who evaluate the design (for certification) that also created the design. But some of the people with certification responsibilities (AR’s) are unfortunately subordinate to management and some who must divulge key information to the AR’s are under career-threatening pressure from company management. Under older delegation arrangements, delegated engineers were directly subordinate to FAA engineers with regard to performance of their delegated tasks. This meant detailed technical oversight that was independent of the company.
The only way for a significant remedy to this chicken/fox paradigm is to eliminate the ODA, and return to the form of delegation that existed 25 years ago.
Unfortunately, there is also an existing core of management within the FAA who were selected during the last two decades with a long history of holding on to their positions of authority by defending the ODA, blinded, intentionally or not, to the fundamental flaws. This “blindness” will resist the changes that are necessary.