Use of simulator time and new ICAO sim standards

[Tee Emm]

The NTSB report on this accident certainly mentions the AA unusual attitude training as a possible contributor.

It would be instructive to read exactly what technique American Airlines published for unusual attitude recovery. Can any reader provide a link? While the NTSB report apparently stated the AA unusual attitude training as a possible contributor, the term "possible" means just that. A good lawyer would make mincemeat of "possible."

Certainly Boeing (737) suggest that cautious use of appropriate rudder pedal input may be necessary to assist in recovering from a nose high low airspeed attitude if elevator or stab trim is insufficient to drop the nose, and a roll input is needed to allow the nose to drop to the horizon.

[downwind]

relavent article;

Civil Simulator Special: Reality check for civil simulators

[lefthanddownabit]

Every circling approach.

These can be done even in simulators without continuous FOV visuals (with certain constraints relating to scene content). Many FAA Level C sims still have monitor visuals and can be qualified for circling approaches. What percentage of sim time does this exercise represent? Does that justify the significant extra cost?

Extending 150 degree visual displays to 180 is achievable cost effectively. A 200 degree requirement (imposed without any real industry-wide discussion and exceeding current FAA and EASA requirements) requires additional visual channels and projectors as well as a new mirror and back-projection screen. It can be achieved but should it be a requirement for all new simulators?

[lefthanddownabit]

It would be instructive to read exactly what technique American Airlines published for unusual attitude recovery. Can any reader provide a link? While the NTSB report apparently stated the AA unusual attitude training as a possible contributor, the term "possible" means just that. A good lawyer would make mincemeat of "possible."

The NTSB report is here:

http://www.ntsb.gov/publictn/2004/AAR0404.pdf

Here's the relevant extract from the training programme as quoted by the NTSB report (page 85).

• Apply climb power
• Maintain 15º to 30º deck angle
• Respect the stick shaker (Fly in the PLI [pitch limit indication])
• Now roll alternately left and right to 40º of bank –

MAINTAIN HIGH AOA

• First, use only ailerons and spoilers
– Note: Sluggish roll response – Developing sink rate
• Second, use only rudder – (smoothly)
– Note: Improved roll response – Developing climb rate
• Third, practice combination (both aileron & rudder)
– Note: Optimum roll response

It's clear a trainee might conclude that rudder inputs are always to be used as roll rates are improved. However this is intended for high AOA situations (close to stick shake), AA587 was accelerating through 255 knots and climbing in a normal attitude so AOA was low.

The NTSB often has to use words like possible and probable. Even lawyers have to accept the concept of doubt. The point made was not that AA's training was at fault, but that a trainee might have applied it in the wrong situation. Testimony about a previous wake turbulence encounter seems to support this conclusion (pages 12-13). So it's hard to see how a lawyer could make mincemeat of the possibllity given the supporting evidence.

The report quotes the initial bank angle deviation as 10 degrees (from the 23 degree banked turn). Not an unusual attitude by any standard.

Certainly Boeing (737) suggest that cautious use of appropriate rudder pedal input may be necessary to assist in recovering from a nose high low airspeed attitude if elevator or stab trim is insufficient to drop the nose, and a roll input is needed to allow the nose to drop to the horizon.

That says it's for nose high, low airspeed conditions. Note also it says "cautious use". Repeated full reversals were employed on AA587. The pilot may well have assumed the rudder limiter would take care of structural issues, but this is not always true for an A300-600. Even if the A300-600 did have a rudder limiter (and not a pedal limiter), the stresses imposed may not have prevented the fin shearing off.

[AirRabbit]

Originally Posted by lefthanddownabit

The ICAO document doesn't replace existing NAA documents such as FAA Part 60 or EASA JAR-FSTD. I'm sure the goal of harmonised simulator design standards is a fine one, but the industry has come close to this before but diverged again. It also throws a few curve balls into the process.

While this is true, at least for now, it is likely that the ICAO member nations will adopt the new standards. However, what some may not know is that adopting the new standards will have no effect on the simulators that currently exist … a process that has come to be known as “grandfathering” … and the new standards would only affect the new simulators coming into the inventory.

Originally Posted by lefthanddownabit

Confusingly it introduces a new set of device types without clearly identifying what they are (Types I-VII). From what I can see Type VII is a full flight simulator, so the document ignores the existence of Levels A thru D devices. It also sets some criteria which are in excess of existing (for example minimum lateral visual FOV is put at 200 degrees when many operators are struggling to meet the existing 180 degree standard set out in JAR-FSTD). That is 50 degrees more than most existing simulators. How much training in simulators requires such a FOV anyway? So the simulator world isn't going to change overnight.

True, the simulator world isn’t going to change overnight – but it IS going to change. It is possible that you haven’t had the opportunity to read the entire set of documents and see the specific requirements including how the Summary Matrix was developed. In short, there were slightly more than 200 piloting tasks evaluated against 12 simulation features, where each feature has 4 levels of fidelity, and each of these combinations were distributed over 15 different kinds of training or license levels, and separated into two categories – training introduction and training to proficiency. This results in a rather massive matrix of just less than 300,000 data points. Obviously, this wouldn’t be manageable by many persons, so once this work was completed, there was a concerted effort to group together those training and license levels that had similar results in the matrix – and where it didn’t make sense to group types of training or licensing requirements together – they were left as a “group of 1.” It was this grouping that generated the 7 types of flight simulation training devices:
Type I includes PPL, CPL, and MPL-1 Core training;
Type II includes training for Instrument Rating;
Type III includes training for a Class Rating;
Type IV includes training (and training to proficiency) for MPL-2 Basic;
Type V includes training for Initial Operator, Recurrent Operator, Recurrent License, and Type Rating;
Type VI includes training for MPL-3 Intermediate; and
Type VII includes training to proficiency for MPL-4 Advanced, Type Ratings, ATPL, Initial Operator, Recurrent Operator, Continuing Qualification, Recurrent License, and training for Recency of Takeoffs/Landings.

The traditional Level A, B, C, and D simulators will continue under the “grandfather” authorizations.

The field-of-view (FOV) requirements were not blindly selected. If you fly an airplane at the designed traffic pattern altitude on the downwind, the location of where you would normally check the runway threshold to begin your turn to base would require you to turn your head just very slightly less than 100 degrees toward the runway. Therefore, for both sides of the cockpit, the required FOV for the visual system would be 200 degrees to fly a VFR traffic pattern. This 200 degree requirement is also just under the logical horizontal FOV for an installation using just 4 visual projectors (going to something like 210 degrees would require a 5th projector). An FOV of just 180 degrees would put whatever 90-degree reference point you may desire right on the very edge of the visual display and any movement might place that point out of view – and the 180-degree system would also require 4 projectors.

Simulators are currently being delivered with 200 degree FOV visual systems rather regularly.

Originally Posted by lefthanddownabit

As has been stated above, unusual attitude and jet upset training requires data which doesn't exist at present. Incomplete data could well result in negative training. Even stall behaviour is not adequately presented in current airliner aero models, nor does it have to be.

While it is true that a sufficient quantity of data has not been routinely collected that will allow an accurate development of a mathematical model of the aerodynamics of a given aircraft beyond the boundaries of a given flight envelope, it is also true that taking the simulator outside of that envelope is not like it used to be. Aerodynamic math models used to be terribly limited – to the extent that rolling the simulated airplane to a bank angle of something like 60 degrees would confuse the programming and the simulator would often freeze in position. Those kinds of circumstances are not seen in simulators built over the past couple of decades. These “newer” aerodynamic models are fully functional through 360 degrees of roll and 360 degrees of pitch. But – like I said – these models have to be modified with specific airplane data to allow the simulator to perform and handle like a given airplane. It is this data that is missing for a completely accurate math model to be developed.

Also, while I said there was not a sufficient quantity of data beyond certain limits, that does not mean that data has not been gathered beyond those limits. Data does exist well beyond the supposed “flight envelope,” but it’s not sufficient to be able to relied upon 100%. So going beyond the “flight test validated envelope” is possible – and while not 100% accurate, it is quite accurate just beyond the limit and gets less so the farther away the simulated airplane is taken from those limits. However, there are some things that are still quite accurate regardless of the attitude of the simulated airplane … the visual display will accurately display what the pilot would see in that attitude; the flight instruments will accurately display what the pilot would read on those instruments with the airplane in that attitude; the performance indications of the change of those instruments (rolling, accelerating, climbing, descending, etc.) would be accurate with regard to what the airplane would be doing in that attitude. Where this lack of data becomes problematic is the aircraft response to given control inputs. The simulator will respond to control inputs to the simulator controls – but we don’t accurately know how quickly the airplane will respond and we don’t accurately know the magnitude of that response. Therefore, we cannot use the simulator to teach or test a specific control application, with the expectation that a pilot experiencing that same circumstance can expect his/her airplane to respond the way the simulator did in those areas.

Another piece of information is that we do have substantial data on aircraft performance and handling qualities all the way down to the stall – that means to the stall break or the controls reaching displacement limits. It would be incorrect to say that going just barely beyond that point puts the simulator into “no man’s land.” But, it is also true that the performance and handling qualities at that point cannot be absolutely guaranteed – it will be close, but not exact.

Originally Posted by lefthanddownabit

UA and jet upset scenarios have been programmed into simulators for decades, but risk negative training due to lack of data and the unrealistic way such upsets must be introduced. An early set of UA scenarios I recall simply put the simulator in a particular condition, frozen, then required the crew to recover when freeze was released. Some ramp in the attitude and the crew are requested to be hands off during this process. You can subtly induce a UA by failing attitude indications, but if the failure is identified the UA is never reached and so no recovery is taught.

Because of these limitations and because of the value of training in a simulator rather than risking exposure in an aircraft, there are teams currently working on how to use what is available in existing simulators, recognizing the limitations that exist, and structuring training programs around the areas that could be problematic yet still providing the crew members with very valuable training on what to expect and what courses of action might be considered in an array of circumstances. Teams are also working on what kind of data might be able to be gathered and what kind of compromises might be available for the structure of an accurate math model where we’ve never had math model accuracies previously.

[lefthanddownabit]

AirRabbit:

I've been in the simulator industry for 30 years. So for the sake of peace and quite I'll ignore the "egg-sucking lessons" you gave me in your lengthy post (I'm well aware of grandfathering, for example):

EASA and the FAA have both invested considerable time and effort in creating their respective approval standards. They aren't going to ditch those in a hurry. FAR Part 60 is part of US aviation law it so will not be so easy to change as ACs used to be. So Levels A-D will continue to be current, not grandfathered, for years to come. There have been attempts to ditch Levels A and C before, both strongly resisted by operators of same.

What may happen is that operators will seek to achieve MPL standards in addition to the meeting the FAA/EASA standards as before.

I certainly did read the whole document and found the definition of the various device types, but it is not presented in a clear or cocnise manner. Type VII effectively equates to a Level D FFS, leaving Levels A-C hanging. Many very capable FFSs exist at Level B and C which seem to be falling through the net here. The change to different terminology when the entire industry quite happily understands the existing terminology, is questionable. Defining simulators by the training that can be done on them is not new either (CAA CAP 453 did this years ago).

Some simulator operators I've spoken too were most dismayed that having got used to and upgraded to the 180 degree FOV, 200 degrees was now being discussed. With FAA and EASA regs, operators are included in the discussions and rule making. The ICAO document appears to have been constructed without the normal levels of discussion. I realise the concept of MPL introduces changes, but these should not force a re-write of existing, satisfactory and well understood standards.

As I said in answer to a previous post, the circling apprach scenario is a relatively small part of the FFS training requirement to hang such a costly change on. Operators can by all means specify such a FOV when they purchase simulators, but should it really be considered the minimum necessary?

[GarageYears]

Ermm, it was Ed Cook of the FAA that initiated the new doc

Lefthanddownabit, I suspect AirRabbit was not directing all of his post directly at you, but (hopefully) the broader audience, but either way, please understand (and perhaps you do already) it was Ed Cook of the FAA that initiated the whole effort to produce this standard.

I believe the whole process was kicked off at an RAeS conference in 2005 - I came on-board during mid-2007. The intent of this effort was both the harmonization of international standards and a revamp of the standards to reflect current technology capabilities. Remember both the FAA Part 60 and JAA JAR-STD 1A ammendment 3 are dervied to greater or lesser extent from the RAeS Aeroplane Simulator Evaluation Handbook 3rd Ed, 2005.

I do take exception to the statement "The ICAO document appears to have been constructed without the normal levels of discussion.". That could not be further from the truth at all! All manner of airline reps, simulator manufacturers, aircraft manufacturers, regulatory authorties and subject matter experts were part of the International Working Group. I can probably dig up the list if that is of interest.

Right now Larry McDonald from the FAA NSP is heavily involved in developing the Helicopter equivalent to the ICAO 9625 Rev 3 document, so the FAA and EASA are not sidelined here, but part of the process, in fact central.

Please see: RAeS ICAO 9625 International Working Group - Index

[AirRabbit]

Hey GarageYears

First, thanks for recognizing that I wasn’t presuming that our friend (lefthanddownabit) was in need of any pointed lectures – of course, if he took any of my comments that way, I certainly want to extend my most sincere apologies. It was only because I have some knowledge in this area that I decided to provide anyone interested with some information on the subject – and in doing so, I was thinking that rather than offer comments in a teasing manner, I’d tell anyone who might be interested what the facts are about this particular subject. I didn’t mean for it to come off like a lecture – just presenting the facts ma’am.

I think you are abolutely correct about the FAA and their intentions. Of course until we actually see something in writing there is always a bit of conjecture about what any regulator is going to do, and the FAA certainly doesn’t have a particular corner on that market.

lefthanddownabit makes some good points about certain simulators currently in service – and no one would want to see them disappear completely. However, with the computer industry expanding like it is, it won’t be long before it will have an even greater impact on the aviation industry – particularly the training parts, and most specifically, those parts having to do with flight simulators. Additionally, keeping older simulators up to spec is becoming increasingly difficult - particularly in light of the falling prices of brand new simualtors and simulator systems. In fact, according to the FAA’s website, out of almost 800 active simulators there are something like 25 – 30 of them that are not Level C or Level D. And, if I’m reading that material correctly, it doesn’t appear that very many Level C simulators are entering the inventory on a permanent basis. It seems that the most sophisticated simulator (Level D) and a couple of levels of the smaller flight training devices, are what is desired most by the training industry – and that, too, is reflected in the ICAO documents. Of the seven types of ICAO devices all but the top two are fixed-base devices – without motion – but all of them have a pretty sophisticated visual system – and all the visual systems will be equipped with at least that same 200-degree FOV presentation we discussed the other day. The lower level devices are designed primarily to address PPL and CPL training – again, probably channeled very much along the lines of the ICAO defined MPL training process.

I also know that one of the reasons that this particular effort was undertaken was that, despite the industry’s familiarity with levels of simulators and flight training devices, there was a wide range of differences between many of them – so the familiarity wasn’t so familiar when it came right down to the specific device. Additionally, without there being an accepted standard, some simulator owners who leased time on a simulator to several organizations from various countries, had to endure the evaluation of that one simulator by the regulatory authorities in each of the respective countries. According to the material on the Royal Aeronautical Society’s website, at the moment, there are apparently some 27 or 28 different levels of simulation that range from table top instrument trainers to the top-of-the-line Level D simulator in service around the world. By any argument 7 levels has to be better than almost 30 – and if all the regulators can get past the personalizations involved, it just might allow one evaluation of a simulator to satisfy all the regulatory authorities’ need to be sure the thing works properly.

[ZFT]

leftanddownabit

As I said in answer to a previous post, the circling approach scenario is a relatively small part of the FFS training requirement to hang such a costly change on. Operators can by all means specify such a FOV when they purchase simulators, but should it really be considered the minimum necessary?

Why is the 180 – 200 FOV a costly ‘change on’? The latest LCoS systems can easily support 200 degrees with a 3 channel system and in fact the existing 180 optics can almost be stretched to achieved 200 as well (during a recent upgrade we experimented with optically stretching the FOV and found we could achieve 197)

With a new build, the delta between 180/200 optics should be negligible.

I would also dispute circling approaches being a relatively small part of the FFS training requirements. This may well be true for some operators who only fly into major international airfields, but there are many other operators who typically operate into airfields with very limited facilities where circling approaches are the order of the day. It is these operators who require extended FOVs, highly detailed databases etc.

[simulation.org.uk]

A very interesting article ...

[A37575]

would also dispute circling approaches being a relatively small part of the FFS training requirements. This may well be true for some operators who only fly into major international airfields, but there are many other operators who typically operate into airfields with very limited facilities where circling approaches are the order of the day. It is these operators who require extended FOVs, highly detailed databases etc

I agree. The present situation for circling approaches in simulators (I am more familar with the 737), is that instead of conducting the circling manoeuvre with the emphasis on watching the runway all the way from downwind to final, pilots are authorised to cheat a little and navigate around the circuit on instruments because otherwise a missed approach would always be the case once you lose sight of the runway as it passes your 90 degree.

I have observed crews building a lovely little picture on the MAP which Picasso would have been proud of, and they LNAV their way around the magenta line with head firmly glued inside. It works beautifully of course and may or may not be a safe policy in real life. I guess it is better than nothing at all, especially where the regulator demands a circling approach form part of the instrument rating test conducted in the simulator and pretends not to notice the crew are heads down typing away on the downwind leg to get the turning point on base entered into the CDU.

Yet I have seen crews castigated for being high turning final when they had no hope of picking the runway when the instructor has set the visibility to the published minimum which under Australian AIP is 4 kms for Cat C types.
On the other hand, if the pilot opts to extend downwind but still stay within the circling MDA safe area of ICAO 4.2 nm, to avoid the risk of getting too high on final, some instructors will say that is cheating!

While a circling approach with low MDA takes more skill than a normal 1500 ft visual circuit, especially if the chart limiting vis is fed into the equation, then surely a wider field of vision in the simulator is a significant factor in terms of flight safety. There is a saying that "get away with anything long enough, and the perceived risk diminishes." So it is with flogging around a circling approach on instruments simply because of a simulator limitation with side vision.

[777den]

On most of the simulator designs I am familiar with going beyond 180 degrees runs in to problems with the simulator structure limiting the field of view. at minimum you are looking at replacing the BP screen and probably the entire mirror cell assembly. The mounting surface the projectors sit on may need replacing and area of the outside of the sim just aft of the mirror cell may need redesign. going from a 3 to 4 projector system means that you end up with an edge blend right in the middle of the primary field of view which is something I personally would try and avoid.

[GarageYears]

The new standard will only apply to new-build simulators, so existing operators of existing devices can leave them as-is if they choose. There is no intent to force the new standard to apply retrospectively - per previous posts those existing devices will retain the current certification under grandfathering rules.

Whether current designs can/do support 200 degree FOV is a problem for the manufacturers. These standards have been available as a draft and in discussion for 3 years or so; so it would be rather hard to claim this was a surprise or news to anyone with a vested interest in this aspect of sim design.

[AirRabbit]

GarageYears you are correct – as has been the policy virtually everywhere around the globe, new standards for simulation apply only to the new simulators coming into service under that new standard. And the 200-degree FOV is becoming rather routine today. Yes, there is an edge blend right down the middle of the visual scene – but the technology is getting pretty darn good – at least to the point where you’d have to know where to look and what specifically to look for in order to determine that there was a blended edge in view.

I recognize the simulator structure issues that 777den referenced, but the new designs in simulators and working with display system vendors are effectively dealing with those issues as well. The real problem is not the horizontal field of view, but the vertical fields of view – particularly an issue for the helicopter simulators because of the very large, almost directly down, field of view those operators would like to have. I think you mentioned in your earlier post that there is an effort underway to update and modernize the helicopter simulator standards as well, and I know that the 60-degree vertical FOV systems that are out there now (which is 20 degrees more than the airplane simulators require) are pretty impressive. The struggle there is going to be whether the helicopter simulators will require collimated systems (like the fixed wing) or if they will require a direct projected system – and some of those direct projected, 60- X 220-degree FOV systems currently in use get pretty fancy with multiple (5 or 6) projectors.

The thing about this that really interests me is that this is the first time that there have been truly international experts working on a common set of circumstances where everyone involved has a vested interest in the correct, successful, and effective result … and I include the regulators in that mix.

[GarageYears]

Can I ask this though - since the users of the simulators are the crew, how many crew are involved in specifying what the simulator supports, and the features/training capabilities these devices provide?

Although the sim manufacturers obviously interact with their customers, can any pilot comment on their direct involvement with the purchase of a new simulator, or is that handled by some training department, that defines what those devices will provide?

In the case of the new ICAO standard were any crew aware of this effort?

- GY

[AirRabbit]

Hey GarageYears

A good question. Yes, there was substantial representation by "crew members." The working group was made up of 10 FSTD vendors and manufacturers, 6 airframe manufacturers, 6 FSTD operators, airlines and independent training centers, 12 industry bodies (two of which were the US Air Line Pilot's Association and the International Federation of Airline Pilots' Associations) and regulators. In addition there were individual researchers, helicopter OEMs and others. Overall, this group consisted of approximately 80 regular members from 16 countries. Beyond this there were 306 individuals registered on the RAeS website - to stay abreast of what was being discussed, to offer their own opinions, and exchange ideas, thoughts, concerns, etc., with others, including those on the "official" working group. The only reason that this method was settled upon was that having a working group that consisted of as many as 306 members would have been completely unmanageable. There was a requirement to provide only a log-on name and password to participate, and, therefore, it would be difficult to identify all of these 306 persons as to what specific aspect of the industry they were representing. However, the "odds" are that a substantial number of those persons were cockpit crewmembers.

Additionally, the workings of this IWG has been presented at 2 WATS conferences, 2 FSEMC conferences, the EATS conference once, 2 APATS conferences, 4 meetings of the Royal Aeronautical Society, and 1 SimTech meeting. Also there were special briefings provided to EASA/JAA (twice), the US FAA (twice), the CAAC (China), as well as several Airlines and Training Centers. Also, the workings were covered in several aviation publications.

Of course, we must also recognize that many of the groups that were represented, although not normally be considered "cockpit crew" members on a regular basis, many of these individuals maintain flight operations (line) currency and would be required to do that through the use of one of these FSTD device types.

In all, it is apparent that this effort has been vetted in just about every way possible - and the results are indicative of thoughtful consideration and many, many hours of review, revision, thought-proving conversations - many of which often were of the "slightly elevated temperature" type - but blows were never exchanged - and the results were eventually agreed to by all parties involved. It is a significantly forward looking and profound work by as large an international workforce as likely has ever taken on such work.

[A-FLOOR]

Interesting article in Flight: FAA proposes new certification category for upset training aircraft

Clearly some prayers were heard...