Final from Dr.R.J.Hall
-----------------------------------------------------------------------------
Appendix B
Basis For Cost-Benefit Analysis of the Extension of ADS-B Beyond Replacement of Radar Like Services.
Dr. R. J. Hall
President, ASAC
10/11/05
ASAC has provided Airservices and ASTRA with a detailed analysis of the ASTRA cost benefit analysis (Appendix B1). The response by the ABIT Chair did not add any information and left our conclusions unchanged. I refer to this document for detail and the following summary is provided here.
The cost benefit study by ASTRA;
“Cross Industry Business Case and Cost-benefit Analysis: ADS-B Avionics Fitment” V1.1 – ASTRA-CIBC 01 ,
reaches a wrong conclusion for two reasons:
1. It assumes fitment of ADS-B IN by GA in calculating the benefits but it does not include the cost of this fitment in the total cost calculation. (Outside radar coverage and third party intervention, ADS-B IN is required for any benefit.)
2. The cost benefit includes cost savings for accident types not requiring fitment of ADS-B OUT by other aircraft. Specifically CFIT and facilitated reductions in SAR times. This is an outcome of fitment in the affected aircraft only. Recommended elective fitment meets this outcome.
This study does show that replacement of some SSR radar with ADS-B is justified as proposed.
The mandate required to meet this proposal is for those receiving a radar service – that is, for all in Classes A and C and for IFR in Class E.
Extension of the mandate beyond that which is required to meet this service provision must depend on the hazard addressed by requiring fitment by all and the cost of this mitigator. Of the accident types discussed by ASTRA only mid-air collision meets these requirements. Discussion by all in meetings of ASTRA and ABIT confirms the view that it is this accident type which drives the push for this mitigator.
ASAC points out that the mid-air collision hazard in Classes E and G is very small. There is no significant hazard en route and the hazard is limited to the terminal area of uncontrolled airfields.
The hazard in the terminal area is addressed by NAS 2c CTAF procedures and depends on radio alerted see-and-avoid. It is also further mitigated by TCAS and transpoders. The size of this hazard does not justify a very expensive third mitigator as follows.
Based on ASTRA figures (see below) the cost of providing an ADS-B alert in this airspace is, over the period of project, $110 million. This is composed of $30 million for installation of ADS-B OUT in additional GA aircraft and an estimated $80 million for an 80% fitment of ADS-B IN in GA.
The benefit achieved, even based on the optimistic figures for the effectiveness of this second mitigator in the terminal area quoted by ASTRA, is $17 Million or 0.8 fatalities saved per year.
Cost – benefit comparison based on the ASTRA report
Figures from the “Cross Industry Business Case and Cost-benefit Analysis: ADS-B Avionics Fitment” V1.1 – ASTRA-CIBC 01
Cost/Benefit of the Extension of ADS-B beyond replacement of radar
Comparison of Scenario B and D – (See Table on page 8 of 39 of the ASTRA report).
Costs
Cost of fitment of ADS-B OUT for GA in Scenario D is $m99 total
Cost of fitment of ADS-B OUT for GA in Scenario B (replacement of existing radar) is $m69
Additional cost of ADS-B OUT for GA $m30
Cost of fitment of ADS-B IN was not quantified by ASTRA.
Assuming the cost and fitment rate of ADS-B IN to be the same as ADS-B OUT this cost was estimated as the same as ADS-B OUT. This is believed to be conservative and estimates the total cost of a single combined installation as $16,000. I have seen estimates of $30,000. ASTRA could be asked to refine this estimate – but we cannot see that this outcome would be significantly altered in terms of the decision supported.
Estimated cost of 100% fitment of ADS-B IN on the same basis as ADS-B OUT $m99
As only one aircraft needs to be fitted a fitment level of 80% would suffice. This leaves a 2% chance of neither aircraft being fitted – which seems a reasonable level.
i.e. Cost of fitment of ADS-B IN for GA = $m80
Total cost of deriving the benefits assumed in the ASTRA report $m110
Benefits
Quantified as fatalities saved by mid-air collisions avoided CFIT accident avoided and reduced SAR times.
Collision accidents avoided 0.8 pa
CFIT accidents avoided 1.56 pa
Reduced SAR times 1.05 pa
Total 3.41 pa
Accidents saved by ADS-B mandate – i.e. accidents requiring fitment in other aircraft
Collision accident only – 0.8 pa
Value of benefit – 3.41 fatalities avoided pa = $m72.2 total – (50.6 to GA and 22.7 to community)
Value of benefit from mandate (collision accidents only) - $m17 ($m72.2 /3.41 x 0.8 = $m17)
Total actual benefit from mandatory fitment $m17
Appendix B1
COMMENTS ON ASTRA ADS-B REPORT
“CROSS INDUSTRY BUSINESS CASE AND COST BENEFIT ANALYSIS”
Dr. R. J. Hall
President, ASAC
20/06/05
SUMMARY
The proponents of ADS-B claim that ADS-B provides the opportunity to improve aviation safety and reduce costs. The cost benefit analysis presented in the ASTRA Report shows unequivocally that this is not true.
Improved safety outcomes depend heavily on extension of ADS-B to regions not currently covered by radar. The central accident type driving these views is a midair collision. This ASTRA report and associated CASA reports show that these accidents occur essentially exclusively in the region of an airfield with more than half in the circuit itself. Application of ADS-B to prevent this accident type depends on a GA VFR pilot monitoring a moving map display in the circuit area. Experience shows that this procedure would significantly increase accidents at airports not reduce them. Standard TCAS procedures requires a commercial IFR pilot to turn off the TCAS warning prior to entry into the circuit to avoid distraction in the this phase of flight.
The other safety outcomes claimed in the ASTRA report do not depend on ADS-B as the appropriate mitigator.
The proposal is far from low cost. The real cost of the combined project is more than $m 300 including some $m 200 (largely by GA) for the extension of ADS-B coverage to achieve the claimed benefits to GA. These benefits are realistically valued at less than $m 25 over the assumed life of the project (19 years). (See below)
The data does show that ADS-B can provide a radar-like service for ATC purposes at reduced cost to Airservices. However, this is core safety technology and the savings achieved (3-5% reduction in the costs of en route services) do not justify exposure to the financial and operational risks associated with a move to ADS-B until there is actual international operational experience and certainty regarding the choice of ADS-B operating system in the US and Europe.
Where ADS-B is used to replace radar, this data and simple logic shows that ADS-B OUT should be mandated based on ATC service requirements. That is ADS-B OUT should be mandated for all in Classes A and C and for IFR operations in Classes E and D. Other existing procedures should apply to VFR for Class D airspace and the NAS recommendations for Class E airspace – see attached.
This approach is not only logical, but it:
a) will impose costs on sectors which require or desire services,
b) is risk management based, and will deliver good safe outcomes,
c) will maximise real return for investment to Airservices and hence introduce real savings to the major airlines,
i) by incorporating savings from both replacement of radar with ADS-B and Nav aids with GNSS based avionics,
ii) and minimise mandated costs to those actually necessary to achieve these outcomes.
and, hence, will maximise reduction in ATC charges for ATC service users.
Security Considerations
If security considerations override this risk management outcome then the existing exemptions for unpowered craft must be retained. This is justified by the draconian effect of imposition of this requirement on these craft combined with the improbability of these craft reaching a centre of population without the assistance of a powered craft.
DETAILED COMMENTS
This analysis needs to consider two projects:
Project 1. The replacement of some existing SSR radar services by ADS-B. This project is largely not controversial but has a decision time line requiring an initial decision by first quarter 2006.
Project 2. The extension of ADS-B services beyond current radar coverage. This project involves very considerable additional investment, is very controversial and does not have a specific decision time line.
Decoupling these projects allows a timely decision regarding the replacement of some SSR radar with ADS-B while allowing time for consultation and for the important controversial decisions involved in this project to be based on real experience with ADS-B.
SAFETY OUTCOMES
The widespread enthusiasm for this project derives from the perceived safety benefits associated with midair collision avoidance. This depends on widespread extension of ADS-B coverage outside current radar coverage – Project 2 (Scenario D in the ASTRA report).
The accident record relied on in this ASTRA Report, backed by extensive analysis and modeling, shows that midair collision hazard exists exclusively in the region of airfields.
CASA examined the Australian midair collision record and concluded that ADS-B would have reduced fatalities by this cause by 80%. Examination of the Australian accidents used to justify the conclusion by CASA shows that all but 2 occurred in the region of an airfield. 14 out of 29 accidents were assumed avoided by ADS-B. 9 out of the 14 collisions assumed avoided occurred in the circuit itself. No allowance was made for accidents caused by inappropriate use of ADS-B.
There have been no collisions involving RPT aircraft and no en route collision in this airspace. Detailed analysis and modeling confirm that this is the expected result – that is, it does not depend on ‘luck’.
Collision avoidance procedures based on ADS-B coverage requires GA to fit both ADS-B OUT and ADS-B IN and a sophisticated moving map display. More importantly, implementation of the information provided to collision avoidance in the region of an airfield requires that a GA VFR pilot monitor that moving map display in the circuit area – ie during take off, circuit and landing.
Because of the distraction caused, current TCAS procedures require that the TCAS warning be turned off when approaching the circuit area.
Experience in glider operations has identified excessive monitoring of GPS improved cockpit displays as a significant factor in a number of recent glider-to-glider midair collision accidents.
Current proposals for revised CTAF procedures at uncontrolled airfields rely on the proven practices used in the US which use an improved alert process which enhances, not replaces, see-and-avoid. These procedures encourage pilots to look outside the aircraft rather than concentrate on an alert from within the aircraft.
The suggestion that a GA VFR pilot should be encouraged to monitor a moving map display during take off, circuit and landing is of very great concern.
Application away from the region of an airfield will have no practical effect as the accident record and extensive analysis and modeling shows that there is no hazard in en route airspace. Estimates of number of aircraft in this airspace show that the maximum number in all of Classes E and G (ie outside existing radar coverage) is conservatively high at ca 300. Ask yourself the following. How much money would you spend to prevent a road collision if there were 300 cars outside the main centres of population? Then remember that cars move in two dimensions not three.
The claims of cheaply obtained improved safety are illusory.
Cost Benefit Analysis
The revised benefit/cost ratio (Table on p 8 of the report) shows that replacement of some SSR radar by ATS-B saves considerable capital investment for Airservices. These benefits to Airservices are highest for Project 1 (Scenario B in the ASTRA Report). These benefits to Airservices are halved in Project 2 (Scenario D in the ASTRA report).
The only significant benefit claimed by this corrected report now lies in the GA sector. The benefits accruing to the other sectors are minor.
(The nature of the errors in the report as first released do not engender confidence and suggest that the rush to decision on this project involves very significant risks which are not appropriate to such a decision. By way of example, the average flight time for a domestic flight was accidentally entered at 10 times the actual value. This error resulted in an inflated return to the larger airline sector. Several other instances of erroneously inflated numbers occurred. This is not to comment on the professionalism of those involved as this is not in doubt. However, it does indicate haste not appropriate to such a decision.)
These comments will firstly examine Project 2 – extension of ADS-B beyond current radar coverage and then some comments on Project 1. More details provided in Appendix B1
C COST BENEFIT of EXTENSION OF ADS-B BEYOND CURRENT RADAR COVERAGE – PROJECT 2
The ASTRA Report claims that the cost benefit analysis is conservative and underestimates the benefit/cost situation. This is untrue.
The benefit/cost outcome for Project 1 (Scenario B in the ASTRA report) is estimated conservatively but the benefit/cost outcome for Project 2 (Scenario D) and to a lesser extent for Scenario C (ASTRA Report) is overestimated thus exaggerating the benefit/cost outcome from the additional investment involved in Project 2. (see table below)
Benefits to GA
The assumed improved safety outcomes are quantified by predicted fatalities assumed saved by mandated ADS-B OUT.
These are summarised in the following table:
ADS-B to current radar coverage Extension of ADS-B beyond current radar coverage Assumed Fatalities
Saved in Project 2 Mandate ADS-B
Accident type Project 1
Scenario B Scenario C Project 2 Scenario D Require ADS-B out Require ADS-B in
Mid-air collision Not included Not included Included 0.8 pa Yes Yes
CFIT Not included 60% fitment Included 1.56 pa No No
SAR Not included Not included Included 1.05 pa No No
Nav Aid Rationalisation Partial Partial Complete No No
1. The revised cost benefit data reported, after removal of errors in the earlier report, shows that the remaining benefits claimed are confined to the GA sector. Benefits claimed for other sectors are marginal at best.
2. The benefits claimed for GA are all indirect and result from assumed improvements in safety.
a) The report assumes an 80% reduction in fatalities due to midair collision. However these collision occur exclusively in the region of airfields. Any effect of ADS-B on this accident type depends on fitment of ADS-B IN, a sophisticated moving map display and rely on an associated recommendation that GA VFR pilots monitor a moving map display during take off circuit and landing. These savings are at best exaggerated and more probably invalid.
b) Avoidance of CFIT accidents depends on a GPS moving map display and does not require ADS B OUT in other aircraft. This outcome will be achieved much more cost effectively simply by the take-up of accurate moving map displays and must be totally removed from this analysis.
c) 35% geographical coverage down to 5000’ is a ineffective means of assisting SAR times. Savings associated with the facilitation of SAR depends on ongoing monitoring of tracks by VFR pilots outside current radar coverage. This is not current practice within radar procedures. No ongoing cost of either manual or automatic monitoring VFR traffic outside of controlled airspace or in Class E is included. This outcome is better achieved using an emergency locator beacon.
The benefits were quantified by the number of fatalities avoided. The ASTRA Report calculated 3.41 pa for Project 2 (full geographic coverage or Scenario D in the ASTRA report) but are more likely zero to 0.4 pa as a result of mandated ADS B OUT – but only then provided this is accompanied with general fitment of ADS-B IN.
3. Rationalisation of, and hence saving from, Nav aids does not depend on an ADS-B OUT mandate. Aircraft dependant on Nav aids need fit C-145/146 TSO GNSS navigators not ADS-B OUT. GA does not need, and the ADS-B system does not depend on, fitment of ADS-B OUT to implement this change. Navigational outcomes are not relevant to the mandate of ADS-B.
4. If Airservices does monitor VFR traffic in Classes E or G then this will introduce a very real problem with duty of care and will introduce ongoing additional costs to Airservices including either or both surveillance costs and/or insurance costs.
Overall
Except for possible benefits from a reduction in midair collisions, the benefits claimed in the ASTRA Report for Project 2 (Scenario D) are not outcomes of mandating ADS-B.
The actual benefit from midair collisions would be more like zero to 0.4 fatalities pa. This outcome would be better addressed by improved application of NAS CTAF procedures and concentration on, and improved training of GA pilots in alerted see-and-avoid and appropriate use of radio.
Costs to GA
Firstly GA has no means of obtaining information for ADS-B unless GA also fit ADS-B IN.
The total cost to GA for Project 2 (Scenario D) is already $m 99 or $m38.6 more than Project 1 (Scenario B). This is for fitment of ADS-B OUT. The actual cost to GA of achieving the outcomes assumed in this report is likely to be more than twice that reported because these depend on full GA equipage with ADS-B IN and a sophisticated moving map display by 2020. The total cost will be something in excess of $m200. The actual additional cost for Project 2 (Scenario D) is then ca $m140.
Conclusion
Project 2 cannot be justified based on the figures reported in the ASTRA Report.
COST JUSTIFICATION OF REPLACEMENT OF RADAR WITH ADS-B – PROJECT 1
Overall, eventual replacement of some SSR radar by ADS-B seems justified for the service provider.
The change moves the major investment from the service provider to GA. The beneficiaries are Airservices and the airlines.
Unless this cost is picked up by Airservices, then this represents a very heavy subsidy of the big end of town by the small end of town which cannot be justified under any circumstances. To hold GA to ransom over this subsidy to achieve an immediate take up is iniquitous and unacceptable.
How ever or when ever this is introduced, equity demands that investment must be made by the service provider.
While the implementation of Scenario B is a matter for the big end of town the following comments seem relevant.
* This is a complete change to the core technology of the business of the service provider to a new and essentially untried technology.
* This change is made for a return which is reported as (only) 3-5% of the cost of en route services.
* This report does not include the normal due diligence for such a change appropriate to even a public company let alone a public utility, such as Airservices, with significant safety outcomes.
* Very significant financial and safety risks are associated with this change which are not addressed.
a) There is no operational experience with this technology and it would be prudent to assume that there will be some implementation problems.
b) Perhaps more importantly, it remains unclear just which of the ADS-B systems under trial will be adopted by the US and Europe. If Australia ends up with an Australia specific system, the cost of avionics in Australia will be high and further development will be limited. It seems likely that Australia would, eventually, have to at least consider a re-fit.
Added to these risks associated with Project 1 is the very considerable ‘duty of care’ risk associated with collection of data on aircraft movements based on a fitment of avionics mandated by CASA.
OVERALL
It is my contention that this investment should be delayed until there is international operational experience with the system and certainty regarding the ADS-B system implemented in the international market, even if this delay would result in significant additional investment to maintain the current system. Given that the NZ SSR radars are older than those in Australia, but otherwise identical, it does not seem that replacement at this time is essential and refurbishment, under the present circumstances, would be a cost effective way forward. If that was impossible then a quantitative financial risk assessment would need to be carried out. I doubt that this would leave much of the 3 – 5% return on the bottom line intact.
SECURITY ISSUES
If this risk management basis for decision is overridden by security issues then, especially given the very large distances between the site of operations and any significant population centre, it is essential that current exemptions from carriage of transponder for unpowered aircraft unable to power these devices (largely gliders – GFA and HGFA – and balloons), based on the absence of ‘an engine driven generator capable of continuously powering such a device’ must be retained and applied to ADS-B. The craft included in this category would be unable to reach a population centre without assistance of a powered craft.