'No blame' Over RAF Tornado Crash

[Safeware]

BGG,

Didn't think there was any thunder to steal - you wanted an opinion from me, duly offered, was just thinking you could give something from your experience - balanced viewpoints and all that.

sw

[Safeware]

Eng(retard),
The HF table was in 00-56 Iss 2 but removed at Iss 3, released Dec 04, so a number of years ago.

sw

[engineer(retard)]

S/W many thanks, was that upgrade to a 2009 or 1909?

BGB

As S/W mentions there were numbers in 00-56 and they have been used. The appropriateness of use is where my original question was aimed. The reason I asked when the change came about, is because I have seen the numbers recently. Now that is probably because of the standards called up when the RFQ was put out. However, in the future it could be expensive to take them out as the whole safety case would have to be re-visited and the answer may not be palatable. Additionally, because the guidance has disappeared, I think that some later safety cases are reaching back and still using the Issue 2 numbers because it is assumed that they are legitimate.

regards

retard

[Chugalug2]

BGG:

Whether this through laziness, ignorance, just plain stupidity, or some combination thereof is open to discussion.

I'm afraid that you've omitted the most likely reason that the numbers game has been played out to one end only, ie to slash safety costs, and that is the one theme that permeates the Mull, Parliamentary, Nimrod threads and now this one. That theme is of deliberate preplanned suborning of the UK Military Airworthiness Regulations by very senior professionals, uniformed and suited. It is a classic case of self regulation gone rotten. The MAA as presently constituted will not change that. This is a monstrous scandal, and all those who have served it should be brought to book. Good people have died and more yet will do so until things change radically. More irritating words I'm afraid BGG, but many more to follow I vow!

[engineer(retard)]

BGG

I am not claiming to be a safety engineer, and have not been an airworthiness signatory for some years now, but I understand your argument.

However, these numbers are being applied in the real world for new designs as an assumption, in lieu of justifiable evidence. As I mentioned in a previous post, I have never seen the assumptions challenged when historical data is available. I personally prefer the approach S/W implies; if you remove the human element, does the system stack up as safe. If not then a critical appraisal needs to be carried out. In my experience, this rarely happens.

regards

retard

[engineer(retard)]

BGG

I think we are mostly in agreement and I understand the cost implications if you looked for a wholly fail safe system. However, I think that many system hazards may be masked by the use of the operator figure. If you remove that element from the hazard analysis it would highlight the weaker areas of design. I'm not suggesting this as an approach to design rather to find a method of turning the tones during independent review.

regards

retard

[Squidlord]

engineer(retard):

When did the human numbers drop out of 00-56, as I have seen them used recently

As S/W mentions there were numbers in 00-56 and they have been used. The appropriateness of use is where my original question was aimed. The reason I asked when the change came about, is because I have seen the numbers recently.

I'm not sure whether you mean:

1. you've seen quantitative human error rates assigned
2. you've seen quantitative human error rates assigned in a way that's either in tune with, or explicitly justified by reference to the scheme in Def Stan 00-56, Issue 2, Part 2

If 1, there's nothing wrong, imo, with assigning quantitative failure rates to humans performing specific tasks in specific contexts. You just have to be very careful not to underestimate the true failure rates . So, the use of quantitative human error rates in a Safety Case prepared in accordance with 00-56 (any issue) is not surprising or necessarily a concern (imo).

If 2, then that might be more worrying if the numbers had been used without validating them (this could just be demonstrating that they are not underestimates) but that may have been done. If it has been done then, again, it's not necessarily a concern (imo).

Additionally, because the guidance has disappeared, I think that some later safety cases are reaching back and still using the Issue 2 numbers because it is assumed that they are legitimate.

I certainly see this happen from time to time. In the same way that some projects are "reaching back" and using the obsolescent Def Stan 00-55 (software safety) because since it was made obsolescent there's otherwise been very little MoD guidance material on software safety (this has changed more recently).



I agree pretty much with everything BigGreenGilbert says in his post 153 (though I'll note that "actual experience" is often not available for the kinds of novel situations and scenarios that MoD often encounters). I'll give my answer to his/her implicit question:

the table [of human error rates] in [Def Stan 00-56, Issue 2, Part 2] is an example, and its inclusion highlights the problems with providing examples. The same way as the HRI tables provided in issue 2 have been misused over the years. Whether this through laziness, ignorance, just plain stupidity, or some combination thereof is open to discussion.

Assuming BigGreenGilbert is referring to the "example" risk matrix and associated definitions in 00-56, Issue 2, I think ignorance is at the heart of its repeated misuse. Ignorance of what a risk matrix (and associated definitions) represents, ignorance of what it means, ignorance of what it should be applied to (given that there are often multiple possibilities) and, most particularly, ignorance of how one might construct an appropriate risk matrix (or matrices, as often more than one is needed).

As it goes, I don't think it's particularly easy to construct an appropriate risk matrix. But if a risk matrix is to be used (it doesn't always have to be), it is such an absolutely fundamental part of safety management that it is totally unacceptable that safety managers, in general, should not have the necessary skills to construct an appropriate one. I think MoD are trying to address this as a particularly aspect of their drive to increase the competence of their safety-responsible staff (not that this will address the issue for industry, of course).



Safeware:

eg lets say in the case of the 43 Sqn Tornado there was a piece of kit that could provide a warning that CFIT was an increasing risk. Lets say it provided warnings to both crew, but had a probability of failure of 1E-3.

It would not be uncommon to see safety arguments that said that this was ok as it required the kit to fail, AND the nav to fail to notice AND the pilot to take notice and take appropriate action. Hence the prob of CFIT would be shown as of the order of 1E-9. The risk of CFIT would therefore be within the airworthiness limit. Right?

Wrong, see my #72:

I think there are potentially any number of issues with the argument Safeware critiques above but I'm not sure what it's got to do with post 72. Are you suggesting, Safeware, that in demonstrating adherence to the JSP 553 cumulative risk target, it's not legitimate to take (quantitative) account of the ability of the aircrew to recover safety from hazards or technical failures (e.g., your CFIT warning failure)?

Safeware again:

As regards the assertion that [safety] arguments are also based on average training and competence, yes I agree with what you say.

I disagree. I think safety arguments should be based on the lowest level of training and competence reasonably foreseeable, for the reasons given in my post 131.

[BOAC]

Putting all these 'statmystics' to one side for a moment, it beggars belief how the same RAF can come to two such wildly different findings on the Mull and this accident.

Was it perhaps that there were fewer careers/knighthoods at stake here, plus, of course, a vastly less polically charged accident?

[engineer(retard)]

Squidlord

The answer is (2). What I am not sure is if they have been used because:

a. It is a documented number that they can stand back from if it goes pear shaped.

b An assumption because no other data was available.

c. Because it makes the safety case add up.

d. Its too expensive to come up with their own data for that particular situation.

That said I am not sure that it is a solvable problem until you have fleet data to work form.

regards

retard

[Safeware]

Squidlord,

No, I don't think you can exclude HF completely, and AMCs for civil Certification Specifications cover 'reasonably anticipated' errors and crew capability. But I don't think 'reasonable' extends to passing the buck to the human because of poor system design and reliance on the human. Def Stan 00-250 Pt 3 is of this view as well:

10.5.11 Combining Equipment Reliability and Human Reliability predictions in Safety Cases Often, the designer must combine equipment reliability predictions (or failure rates) with similar predictions for human actions informal documents such as HAZOPs, fault trees analysis, event tree analysis, probabilistic safety analyses, etc. The foregoing arguments illustrate the difficulties in qualitatively different sets of data.

In such cases, the designer should identify a range of bounding values for Human Reliability, and should initially use such data to identify critical human activities and the sensitivity of the system to human actions. Where system safety is critically dependent on human actions, the designer should aim to reduce the system sensitivity (i.e. produce an error-tolerant design) through iterative design, and to incorporate suitable Workspace / workload and automation strategies to prevent predictable human performance limits being exceeded (e.g. appropriate allocation of functions).

Specialist advice is required where automation strategies are likely to impact on mission or safety critical functionality.

As for defining a safety case in terms of the lowest level of training and competence, I think that is ineffective. If the safety case and RTS for, say, Typhoon, was based on the ab-initio FJ jock on his first sortie, life would be rather restricted. Instead, have a design safety case, airworthiness argument (and RTS) based on the average pilot and then use the management of safe operation to provide the ab-initio with the required comfort blanket of supervision, sortie allocation, currency etc etc to build up/ maintain the skills required. Unfortunately, it seems clear that this is one of the failures in the F3 case.

sw

[Chugalug2]

BOAC:

...it beggars belief how the same RAF can come to two such wildly different findings on the Mull and this accident.

It's notable certainly, BOAC, but quite understandable I'd say. From what we know now it seems likely that the Chinook HC2 was knowingly forced (what other word for it, given that Boscombe's urgent pleas to not do so were blatantly ignored?) into RAF service in a Grossly Unairworthy condition. Given that it was a mere matter of months later that Mull killed 29, including the cream of the UK's anti-terrorism front line, and that the cause was obscured by lack of evidence (no ADR, CVR or direct witnesses) let alone a 'briefed' BoI that ensured no circumstantial evidence would muddy the water, the need and opportunity to nail the pilots for it was clear to those so implicated. That the nailing was so OTT and incapable of standing up to informed scrutiny is a comment on the arrogance and incompetence of those who wielded such a sledge-hammer to this particular nut! This accident by contrast may well raise similar questions of decisions taken by the RAF Higher Command, but more of poor judgement (ie what are the safe minimum monthly hours for a Typhoon AD pilot?) rather than the Gross Negligence that lead to Mull (and not by the pilots!).

[Squidlord]

Safeware, thanks for the reply. I understand where you're coming from wrt the combination of system (excluding humans) safety and HF (and agree, for what it's worth). I guess I was maybe mislead somewhat by your choice of example (and I'm still not sure how all this relates to post 72). After all, TAWS/GPWS type systems are really only backups to the main protection against CFIT. And that's the pilot (right?). So, it would seem odd to shift all responsbility for avoiding CFIT onto some technical gizmo - some of that responsibility surely must stay with the pilot, which would suggest it's not only legitimate but essential to take the aircrew abilities into account when determining whether the risk of CFIT is acceptable. I doubt if any TAWS could be engineered to a sufficiently high standard to provide the necessary protection to allow the aircrew to stop worrying about CFIT, as it were (I know you didn't claim that). And given that we have lots of aircraft flying around, apparently safely enough, without any kind of TAWS, it suggests that, in principle, even a relatively unreliable (1E-3) device like the one Safeware suggests would just make things safer (not necessarily true, of course, if the aircrew come to rely on it too much).

As for what you say about average vs. least level of pilot competence, I think we disagree but I'm not even sure now. And if I try and explain, I think I'll just expose my ignorance of how aircrew are trained, when they are considered to be trained, etc.

Incidentally, changing subject, what on earth does this mean (from Def Stan 00-250, Part 3, 10.5.11):

Often, the designer must combine equipment reliability predictions (or failure rates) with similar predictions for human actions informal documents such as HAZOPs, fault trees analysis, event tree analysis, probabilistic safety analyses, etc.

Is it just me (that can't parse this) or is this another case of hopelessly low quality control on what is supposed to be an authoritative MoD document (as in my post 130).

[The B Word]

Just been thinking about this and ZE982 was the 13th F3 to be totally lost (ie. Cat 5) - unlucky for some .

1. 21 Jul 89 ZE833 CFIT 30nm North East Newcastle
2. 21 Oct 93 ZE858 Catastrophic fuel leak near A66 near Barnard Castle
3. 7 Jun 94 ZE809 Catastrophic engine failure 60nm East of Newcastle
4. 8 Jul 94 ZH558 CFIT near Akrotiri
5. 10 Mar 95 ZE789 Catastrophic engine failure near Donna Nook
6. 30 Oct 95 ZE733 Midair near Boulmer
7. 10 Jan 96 ZE166 Midair near Cranwell
8. 10 Jan 96 ZE862 Midair near Cranwell
9. 28 Sep 96 ZE759 MOD(PE)/BAESYSTEMS aircraft
10. 5 Jun 98 ZE732 CFIT near Flamborough Head
11. 17 Nov 99 ZE830 Catastrophic engine failure near Torness
12. 14 Oct 05 ZE962 LOC near Leuchars
13. 2 Jul 09 ZE982 CFIT near Glen Kinglas

Unless anyone else can correct me?

The B Word

[Pure Pursuit]

I remember 3,4 and 6 rather vividly.

Can't think of any others B word. The frightening thing about no.13 is that there was very nearly a no.14 right behind it. Thank god Blacksmith 2's front seater rammed that throttle forward when he did...

[The B Word]

Yes, I guess they both owe Kenny and Nige a few beers with St Peter for that...

[soddim]

The F3 might have been the RAF's 13th but it was not the 13th F3 loss because the RSAF lost two in a mid-air and one off the runway on landing.

[Fox3WheresMyBanana]

Sobering to think I have half of them in my logbook. ZE 789 was IIRC always in the shed with something broken, and ZH 558 was brand new in 1993.

[Safeware]

Fox3, re 558
It was the last F3 delivered, the fastest (so I heard) and hadn't even had a primary IIRC!

sw

[Fox3WheresMyBanana]

I thought that 559 was the last? Either way, I think it was the AOC that nabbed the last delivery flight. I delivered 556 and 557, after very careful walkrounds. They were digging major components out of the training shops to build the last few. Can't say I was upset about not getting the last one.
I remember the time it took for Manchester to approve a climbout heading of "vertical".

[Safeware]

F3, did a search - sad eh? Apparently both delivered 24 Mar 93!

http://www.tornado-data.com/Producti...ion%20List.pdf

sw