Offered with no intent to be argumentative or accusatory - merely to provide yet another view - accurate by my training and experience, but not to suggest anything beyond that.
What is certainly true today is that under current EASA and I assume FAA regs, there is still no requirement for any objective testing of motion systems (other than technical performance, vibs and latency). Every motion cue on a current Level C/D simulator is still only assessed subjectively!
In that it was never a requirement for me to do so, I have not (as yet anyway) found it necessary to become familiar (let alone, intimately familiar) with the requirements levied on simulator users/sponsors by EASA, or any other regulatory outside of the US. But, I think it very likely to be misleading, at least, to indicate that there are no requirements for any objective testing of motion systems beyond those of vibrations and latency.
First, the testing for vibrations is not merely to see if the simulator can produce vibrations at the points and under the circumstances where the airplane is known to vibrate and do so within some established time frame. Of course it is true that the simulator must provide for the recording of the motion system response time(s) and a Statement of Compliance and Capability (SOC) is required for each. This SOC is a declaration that a specific requirement has been met, explaining how the requirement was met (e.g., gear modeling approach, coefficient of friction sources, etc.) and must also describe the capability of the FSTD to meet the requirement, including references to sources of information for showing compliance, rationale to explain how the referenced material is used, mathematical equations and parameter values used, and conclusions reached.
Additionally, with respect to vibrations, the simulator must provide characteristic motion vibrations that result from operation of the airplane if the vibration marks an event or airplane state that can be sensed in the flight deck. The simulator is to be programmed and instrumented in such a manner that the characteristic buffet modes can be measured and compared to airplane data. That is a description of an “objective” test.
Basically, the motion system tests outlined in the FAA regulations are intended to qualify simulator motion cueing systems from a mechanical performance standpoint. Additionally, the list of motion effects provides a representative sample of dynamic conditions that should be present in the flight simulator. An additional list of representative, training-critical maneuvers, which are to be selected from performance tests, and handling qualities tests, are recorded during initial qualification (but without tolerance) to indicate the flight simulator motion cueing performance signatures have been identified, and are intended to compare with similar results from the airplane and from previous tests of the simulator itself. This was formulated to help improve the overall standard of simulator motion cueing, and to assist in ensuring that this cueing does not deteriorate over time.
The Motion System tests in the FAA rules are intended to qualify the simulator’s motion cueing system from a mechanical performance standpoint. Additionally, the list of motion effects provides a representative sample of dynamic conditions that should be present in the flight simulator. An additional list of representative, training-critical maneuvers, selected from Performance tests and Handling Qualities tests, that are to be recorded during initial qualification (but without tolerance) to indicate the flight simulator motion cueing performance signature has been identified for each of those points. These tests are intended to help improve the overall standard of FFS motion cueing.
Motion System Checks, such as frequency response, leg balance, and turn-a-round check, as described in the FAA rule, are to demonstrate the performance of the motion system hardware, and to check the integrity of the motion set-up with regard to calibration and wear. These tests are independent of the motion cueing software and should be considered robotic tests.
Motion System Repeatability tests are to ensure that the motion system software and motion system hardware have not degraded or changed over time. This diagnostic test is to be completed during continuing qualification checks in lieu of the robotic tests. This will allow an improved ability to determine changes in the software or determine degradation in the hardware. The following information delineates the methodology to be used for this test.
(1) Input: The inputs should be such that rotational accelerations, rotational rates, and linear accelerations are inserted before the transfer from airplane center of gravity to pilot reference point with a minimum amplitude of 5 deg/sec/sec, 10 deg/sec and 0.3 g, respectively, to provide adequate analysis of the output.
(2) Recommended output:
(a) Actual platform linear accelerations; the output will comprise accelerations due to both the linear and rotational motion acceleration;
(b) Position of motion actuators.
Motion Cueing Performance Signature tests are designed to document and verify the following.
(1) The intent of this test is to provide quantitative time history records of motion system response to a selected set of automated QTG maneuvers during initial qualification. This is not intended to be a comparison of the motion platform accelerations against the flight test recorded accelerations (i.e., not to be compared against airplane cueing). If there is a modification to the initially qualified motion software or motion hardware (e.g., motion washout filter, simulator payload change greater than 10%) then a new baseline may need to be established.
(2) The conditions identified in the rule are those maneuvers where motion cueing is the most discernible. They are general tests applicable to all types of airplanes and are to be completed for motion cueing performance signature at any time prior to or during the initial qualification evaluation, and the results included in the master qualification test guide – as benchmarks.
(3) The motion system should be designed with the intent of placing greater importance on those maneuvers that directly influence pilot perception and control of the airplane motions. For the maneuvers identified in the rule, the flight simulator motion cueing system should have a high tilt co-ordination gain, high rotational gain, and high correlation with respect to the airplane simulation model.
(4) The minimum list of parameters that are to be recorded should allow for the determination of the flight simulator's motion cueing performance signature for the initial qualification evaluation. The following parameters have been accepted as sufficient to perform such a function:
(a) Flight model acceleration and rotational rate commands at the pilot reference point;
(b) Motion actuators position;
(c) Actual platform position;
(d) Actual platform acceleration at pilot reference point.
Motion Vibrations are not merely “to be present.”
(1) Presentation of the characteristic motion vibrations may be used to verify that the flight simulator can reproduce the frequency content of the airplane when flown in specific conditions. The test results are to be presented as a Power Spectral Density (PSD) plot with frequencies on the horizontal axis and amplitude on the vertical axis. The airplane data and flight simulator data are to be presented in the same format with the same scaling. The algorithms used for generating the flight simulator data should be the same as those used for the airplane data. If they are not the same then the algorithms used for the flight simulator data should be proven to be sufficiently comparable. As a minimum, the results along the dominant axes should be presented and a rationale for not presenting the other axes should be provided.
(2) The overall trend of the PSD plot should be considered while focusing on the dominant frequencies. Less emphasis should be placed on the differences at the high frequency and low amplitude portions of the PSD plot. During the analysis, certain structural components of the flight simulator have resonant frequencies that are filtered and may not appear in the PSD plot. If filtering is required, the notch filter bandwidth should be limited to 1 Hz to ensure that the buffet feel is not adversely affected. In addition, a rationale should be provided to explain that the characteristic motion vibration is not being adversely affected by the filtering.
The amplitude should match airplane data as described below. However, if the PSD plot was altered for subjective reasons, a rationale should be provided to justify the change. If the plot is on a logarithmic scale, it may be difficult to interpret the amplitude of the buffet in terms of acceleration. For example, a 1×10−3 g-rms2/Hz would describe a heavy buffet and may be seen in the deep stall regime. Alternatively, a 1×10−6 g-rms2/Hz buffet is almost not perceivable; but may represent a flap buffet at low speed. The previous two examples differ in magnitude by 1000. On a PSD plot this represents three decades (one decade is a change in order of magnitude of 10; and two decades is a change in order of magnitude of 100).
All of the above should be sufficient to refute the idea that motion cues on a current Level C or Level D simulator is assessed only subjectively.
ICAO 9625 Edition 3 does address this issue with both frequency and time domain objective testing BUT neither EASA nor the FAA have any plans to adopt Ed 3 in the foreseeable future.
While the official FAA response was somewhat clouded when delivered by former FAA Administrator Babbitt, at the Annual RAeS International Flight Crew Training Conference, in September, 2011, where, from the podium, he said the following:
On the subject of flight simulation training devices, let me address some questions we have received about our position on ICAO Document 9625 (Manual for Criteria for the Qualification of Flight Simulation Training Devices). We appreciate the ground-breaking work that ICAO and the Royal Aeronautical Society have done in developing this document, and we support the principles and standards it contains. We share the view that those principles and standards will lead to significant improvement in simulator training worldwide, and the FAA is working to adopt them.
The timeframe for adoption is the challenge. The magnitude of the changes recommended in 9625 is significant, and many will require rulemaking. And, as you probably know, the FAA’s rulemaking resources and priorities are currently driven by the requirements Congress mandated last year in the Airline Safety and FAA Extension Act of 2010. But we are committed not only to looking at adoption of the principles and standards in 9625, but also to all manner of new technologies and new methodologies for improved flight crew training.
The specific reason there has been little movement in the direction Administrator Babbitt had indicated is 2-fold: first, Mr. Babbitt is no longer the FAA Administrator; and second, there is a small faction of former and current managers who desire to eliminate, or drastically reduce, the requirement for simulator motion from the requirements of a simulator for a proficiency check or for the ATPC certification check. There are also many FAA Safety employees who are mightily fighting this nonsense, but without any outside look, this ridiculous attitude may not be seen for the stupidity that it really is.
I’m not going to repeat myself on the fidelity issue other than to reiterate that there really is no excuse with todays Level C/D FSTDs for anything less than perfection with systems simulation and performance within the normal envelope (within the confines of the obvious technical limitations).
Unfortunately, this statement borders on suffering an internal, self-defeating conflict. The “obvious technical limitations” are what is limiting in some cases and preventing in others, the “perfection of systems simulation and performance.”
There will be an issue with older FSTDs and hard and/or costly choices will need to be made. Excuses such as “data” or (being) “against our training policy” are not limitations. The FAA notice, whilst welcome will not in reality do anything to address the majority of the concerns raised here and with all the regulatory agencies being so ponderous it will not affect any changes until the next decade anyway.
I so wish that the “ponderous” nature of regulatory achievements was not so “ponderous.” But, for whatever it is worth, having some built-in slowness in such endeavors, is probably a good thing, lest we wind up being “hoisted by our own petard.”
Even still, the regulatory requirements that I recognize are far more specific and far more productive toward achieving the kinds of things that make simulation good. Yes, of course, we have further to go … and initiating that movement cannot start earlier than now … I for one am going to attempt to provide information and interest to the persons who may be able to more than acknowledge the existence of todays’ somewhat “menial” standards – at least with respect to where it is I think we all would like to have them … and have them around the world. Also, I would encourage anyone here to do whatever they can to push this effort forward … talk to your management, talk to your fellow pilots, talk to your regulators, get your union involved, write letters or make phone calls to ICAO, to the RAeS, whatever thing you can and are willing to do.
There is an old, old saying that … “for evil (and, here I’d exchange “evil” to “mediocrity”) to triumph, it takes only good men to do nothing.”
Better initial testing, more rigorous evaluations by both the operator and the qualifying body and the end User being more prepared to write up any defect not just accept them would go a long way.
No one in their right mind would challenge any aspect of this statement.