Ferris,
Thank you for your question. We don’t mean to lecture here – neither be patronizing - but the background below may help you and other readers to better understand the true nature of safety case and hazard identification and management.
A safety case is, in its most simplistic form, the recording of a number of structured actions that have been taken to provide assurance as to the safety of a design or a model or the implementation thereof. A safety case is – or should be - a clear demonstration of a robust process of risk assessment and treatment strategies. It may not – and most often does not – identify all of the risks associated with a project – but well conducted, with appropriately qualified people participating in the hazard identification process, it will identify those risks that are most likely to be present in the system. The ongoing nature of a safety case means that once identified and recorded, hazards must be regularly reviewed, the effectiveness of the hazard treatments examined, hazards removed as they are superceded, and other hazards added and treated as they arise.
What is more relevant to your question is the effectiveness of the determination of hazards that may contribute to, or increase the levels of, risk associated with the project – and then determination and assessment of the available options for reducing those risks to “as low as reasonably practicable – ALARP”.
First, the hazard identification process needs to be carried out “without fear or favour”. That is, the participants within any group examining risks should be free to identify and record ANY hazard that they feel is appropriate. That said, the hazards must be able to be clearly linked to the major hazard, which, in the case of aviation, is most likely the “potential for a collision between aircraft”.
So the process must be one that says: “event x could occur, which would lead to an increase in the potential for a collision between two aircraft” – not “event x could happen which I don’t like”, or "event x could happen which would increase our charges", etc.
It is desirable - but not essential – that the identifiers of a hazard then identify activities or actions, which – in their opinion – may bring about a reduction in that hazard. Here again, such mitigators should be suggested without fear or favour. It is not appropriate to dismiss any mitigation at this point – even if it is obviously unlikely to be implemented. In some cases, it may not be possible for the parties present to develop a mitigation. If other parties determine the mitigators independent of the hazard jury, the hazard jury should be given the opportunity to examine the mitigators, and comment on their suitability.
At this point, the suggested mitigators can be evaluated to determine both their cost, and benefit. Most certainly, if a mitigator can be introduced at relatively low cost, WITHOUT substantially altering the characteristics of the change proposed, then it should. If a mitigator is expensive, and the cost would be disproportionate to the benefit – AND the residual risk is within the ALARP range – AND the responsible authorities are prepared to accept the risk – then the mitigator may be discarded – BUT the it must be stated within the safety case that a mitigation was considered and abandoned. In fact the reason for not applying any particular mitigation MUST be recorded.
It is NOT appropriate that hazard mitigations be limited by the convenors of the hazard identification process – even if it is clear that only certain options may be available.
Now to your question regarding effectiveness.
Having identified a mitigator, a process must be put in place – and its effect measured – preferably BEFORE – but most certainly immediately after implementation. That is why we made the point that simply identifying training and education as a mitigator – and apparently a primary mitigator at that – and then dispatching the material – does NOT fulfill the obligations of the safety case. The authors of the safety case MUST be able to demonstrate that the training and education material ACHIEVED THE DESIRED EFFECT.
How is that measured? In the identification of the hazard, it is important that the hazard is correctly specified, so that the causal chain can be seen, and the expected effect of the mitigation seen and recorded. So “event x could happen, …- if we educate the pilot about the hazard and its impacts, then educate the pilot about actions that would reduce the likelihood, then the potential for this hazard reduces”.
This allows two things to occur. First, it is possible to tell the training specialists exactly the event for which the training material is required – i.e., specific hazard related training rather than glib
PR material.
Second, it is possible to test the effectiveness of the material by examining pilots, checking their responses, and determining if the education material is “fit for purpose”. This is usually done with a pilot study, or with a sample group of affected staff before material is released into the public domain.
In all, even for a relatively minor change, we believe that with over 50,000 pilots in Australia, you would need at least to 3 months to develop, test and evaluate, and then deliver training material – AND examine and record IN THE SAFETY CASE that it has been delivered effectively. Adjustments to training material for major change may require the training period to be extended to 6 months or more. Your regulator should only allow a go/no go decision date to be passed ONLY if there is evidence that training and education has been delivered and recorded – or can be delivered and recorded before the start date. This latter situation would probably only occur if there was evidence of prior compliance.
We hope this helps.