News release
TSB completes its independent review of the ATSB’s investigation methodologies and processes
Gatineau, Quebec, 1 December 2014 – Today, the Transportation Safety Board of Canada (TSB) released its
independent peer review of the Australian Transport Safety Bureau’s (ATSB) air investigation methodologies and processes.
The ATSB requested a review after its investigation report (AO-2009-072) into the November 2009 ditching of a Westwind 1124A aircraft near Norfolk Island, Australia, received public and political criticism. The review did not include an actual reinvestigation of the Norfolk Island occurrence, but rather examined how that investigation, along with two other investigations similar in scope, was conducted.
“We saw the potential for mutual learning when we accepted the review request,” said Kathy Fox, Chair of the TSB. “We will now examine if any ATSB investigation best practices should be adopted here in Canada.”
The TSB review compared the two organizations' methodologies against the standards and recommended practices outlined in Annex 13 to the International Civil Aviation Organization and found they met or exceeded the intent and spirit of those prescribed.
However, while there were some ATSB best practices identified in the review, the TSB also found that there was some room for improvement, and has made
14 recommendations in the following four areas:
- Ensuring the consistent application of existing methodologies and processes
- Improving investigation methodologies and processes where they were found to have deficiencies
- Improving the oversight and governance of investigations
- More effectively managing communications challenges
“We have communicated our findings and recommendations to the ATSB for their consideration and action as appropriate,” added Ms. Fox. “Sharing our experiences and expertise is part of our commitment to advancing transportation safety, and when called upon by international partners, the TSB is prepared to assist when feasible.”
The TSB is an independent agency that investigates marine, pipeline, railway and aviation transportation occurrences. Its sole aim is the advancement of transportation safety. It is not the function of the Board to assign fault or determine civil or criminal liability.
For more information, contact:
Transportation Safety Board of Canada
Media Relations
819-994-8053
The Norfolk Island investigation
The TSB Review of the Norfolk Island investigation revealed lapses in the application of the ATSB methodology with respect to the collection of factual information, and a lack of an iterative approach to analysis. The review also identified potential shortcomings in ATSB processes, whereby errors and flawed analysis stemming from the poor application of existing processes were not mitigated.
First of all, an early misunderstanding about the responsibilities of the Australian Civil Aviation Safety Authority (CASA) and the ATSB in the investigation was never resolved. This led to the ATSB collecting insufficient information from Pel-Air to determine the extent to which the flight planning and monitoring deficiencies observed in the occurrence existed in the company in general.
Poor data collection also hampered the analysis of specific safety issues, particularly fuel management, company and regulatory oversight, and fatigue (the ATSB does not use a specific tool to guide investigation of human fatigue).
Weaknesses in the application of the ATSB analysis framework resulted in those data insufficiencies not being addressed and potential systemic oversight issues not being analyzed. Ineffective investigation oversight resulted in issues with data collection and analysis not being identified or resolved in a timely way.
All three peer reviews conducted on the Norfolk Island draft report identified issues with respect to factual information, analysis, and conclusions. Many of these concerns were never followed up after the review process was complete. The ATSB process does not include a second-level review to ensure that feedback from peer reviews is adequately addressed.
After investigation reports have gone through peer and management review, they are sent to directly involved parties (DIPs) for comments. In the Norfolk Island investigation, the DIP process was run twice: once when the report was in its initial draft, and the second time after it had been revised. However, there is no process to ensure that the ATSB communicates its response to DIPs' comments. Formal responses to DIPs increase their understanding of the action taken in response to their submissions, and may make them more amenable to accepting the final report.
In the Norfolk Island investigation, the Commission's review of the report took place immediately after the first DIP process was completed, 31 months after the occurrence. At this stage in an investigation, it is difficult to address issues of insufficient factual information since perishable information will not be available and the collection of other information could incur substantial delays. At the ATSB, the Commission does not formally review some reports until after the DIP process is complete, and in any event, there is no robustly documented process after the Commission review to ensure that its comments are addressed before the report is finalized. Both of these aspects of the ATSB review process increase the risk that deficiencies in the scope of the investigation and the quality of the report will not be addressed.
A safety issue was identified in this investigation concerning insufficient guidance being given to flight crews on obtaining timely weather forecasts en route to help them make decisions when weather conditions at destination were deteriorating. When the safety issue was presented to CASA, it was categorized as “critical”, but in the final report it was described as “minor”, which caused significant concern among stakeholders. The TSB Review observed that this shifted the focus of the discussion to the label and away from the issue itself—and the potential for its mitigation.
In the final stages of the investigation, senior managers were aware of the possibility that the report would generate some controversy, but communications staff were not consulted and no communications plan was developed. Once the investigation became the subject of an external inquiry, the ATSB could no longer comment publicly on the report, which hampered the Bureau's ability to defend its reputation.
The response to the Norfolk Island investigation report clearly demonstrated that it did not address key issues in the way the Australian aviation industry and members of the public expected.
Comment:
BOLLOCKS!
But then we all expected that didn't we...
But after this triple load of bollocks I'm definitely taking the TSBC off my Xmas card list...
The Kangaroo Valley and Canley Vale investigations
The review of the Kangaroo Valley and Canley Vale investigations showed that when the ATSB methodology is adhered to, and the component tools and processes to challenge and strengthen analysis are applied, the result is more defensible.
In contrast to Norfolk Island, the Kangaroo Valley and Canley Vale investigations underwent regular critical reviews and used the ATSB analysis tools effectively, which gave rise to well-documented decisions, and revised data collection plans and analyses. In the Canley Vale investigation, additional information collected as a direct result of a critical review guided informed decisions with respect to the investigation of regulatory oversight.
In the Kangaroo Valley investigation, the target timeline outlined in the ATSB Safety Investigation Quality System (SIQS) for a Level 3 investigation was exceeded, despite significant effort by the team to expedite the investigation. This may indicate that these targets are unrealistic, the investigation was incorrectly classified, or that other work had influenced the published investigation schedules. Significant delays in completing an investigation increase the risk that stakeholders' expectations with respect to timeliness will not be met.
Nevertheless, because of the teams' active engagement with stakeholders in the Kangaroo Valley and Canley Vale investigations, expectations with respect to schedules were well managed and timely action was taken on safety issues.
In the Canley Vale investigation, events prior to the occurrence raised questions with respect to regulatory non-compliance and oversight. The report states that issues of regulatory non-compliance did not contribute to the occurrence, and the analysis tools were indeed effectively used to support this. However, the report could have benefitted from a more thorough discussion to clarify the underlying rationale for this conclusion.
Unlike Norfolk Island and Kangaroo Valley, the Canley Vale investigation included a closure briefing, which provided an opportunity to discuss lessons learned.
Absolutely undeniable clap trap...
MTF...
