Sunfish - I think you make some very valid and important points that are worth thinking about over and over again. There is a lot there to unpack.
Indeed, the kind of data collection done in such programs lends itself very nicely to such statistical analysis. I understand and know exactly what you are saying.
That acknowledged, some considerations are in order. First, one would expect that a statistical approach is one part, perhaps a large part, but not the only part of a thorough approach to flight data analysis. We know this untuitively even though most outside the cockpit of an airliner do not.
I say this, because a statistical approach can have unintended responses and can mask "once-of's" which may highlight a latent problem but which occur rarely.
For example, in trying to advance our flight data program, some non-flying, non-safety managers and executives have suggested (and so far, their views have prevailed, much to our great frustration and concern), that only a "representative sample" of airplanes need be equipped with QAR recording equipment.
Further, they have suggested that one fleet type is a sufficient sample of the operation and further installations are all but unnecessary or at least should wait. It has also been suggested that further installations weren't needed until they could be shown to be "commercially viable". You could have heard a pin drop...
Such a response is financially driven of course and has nothing to do with thinking about flight safety. "Not knowing" for a company's leadership in today's environment is a demonstrably high-risk approach to both financial and flight-safety due diligence but it flourishes as seen, nonetheless.
Also, (and I suspect you know this intuitively), statistics such as averages, means and deviance are right out the window if "you're it" and you've had a crash, (or worse, are in one). There is no such concept as an "average" or "risk", after an accident. It is antecedents, pathways, "why-because" analysis and "was it in our data?" from that point on. Much more could be said...
The other mask which statistics may apply in an unintended fashion is, what if your pilots are simply very good at "rescuing" non-stabilized approaches and achieving successful, touchdown-zone landings 100% of the time? Indeed, some non-flying (and some flying!) managers ask, "If you're saying these approaches are 'high risk', where are the over-run accidents?"
The notion of "
stochastic" may be useful in thinking about both the great value of flight data analysis as well as it's "problems". The term means, very roughly (because I am neither a mathematician or a statistician),
"(Greek stochazein, to shoot with a bow at a target; that is, to scatter events in a partially random manner, some of which achieve a preferred outcorne). If a sequence of events combines a random component with a selective process so that only certain outcomes of the random are allowed to endure, that sequence is said to be stochastic." - Gregory Bateson, Mind and Nature, ca 1960
An example would be, (keeping in mind how accidents happen in our industry), a person running across a fairly busy road or non-rushhour freeway will likely make it first time, second and perhaps a dozen times. But ultimately, done enough times, that person is going to be hit. There is both randomness and a "preferred" (by the process, NOT by the person!), outcome. THAT it is going to happen, of that we are certain. WHEN, cannot be said, nor can flight data tell us which approach will end up as an accident.
In doing their "best" for the organization so that it and it's owners "prosper", the ubiquitous bean-counter, senior airline executives, non-flying managers and those managers who fly but who have lost their way on the way to management, use this sleight-of-thought all the time to justify cost-cutting in flight safety areas and programs. We produce nothing measurable so can it be so bad to cut, or with-hold supporting resources?
Will flying 182kts or 178kts get you a "pull-up and go-around" from ATC? Will 160/4 make you safer? Obviously it depends on a lot of things, very few of which are secure predictors of an accident.
The key in any risk-intensive enterprise is finding the balance and using what data is available intelligently. While this is a black-and-white view which demands a far more subtle approach, at
precisely the time when their services and their work are needed most, these days flight safety departments and their programs are seen as expensive impediments whose conservatism is counter to "efficient" use of resources such as fuel, people and airplanes. The natural tension between these two forces is in danger of being overtaken by commercial side.
The call for SOP adherence throughout this thread is both freshening and heartening, for again, if ever there was a time....
This is one of the most important threads on PPRuNe at the moment. It goes to the heart of what makes this business, in all its aspects, so fascinating.