Simulator visual models

[slast]

Does anyone have any knowledge or expertise in how most simulator visual systems model slant visual range in various non-homogeneous conditions e.g. blowing snow, sand, patchy fog etc? And what percentage of sims in everyday use are equipped to do this? Any pointers to authoritative sources please!

[deptrai]

may I ask for what purpose you are asking that question? It could help to understand what you are aiming at.

"patchy fog"....modern visual renderings attempt to model clouds and fog as particle masses, generated from a database of different cloud types, with some added pseudo-random "growth", and then calculate the interaction with light sources, light absorption, scattering, shadows; and occlusion of objects behind as seen from point of view

blowing snow or sand can be modeled in the same way, but to make a wild guess (because I didnt pay attention to that), accurate generation of blowing sand visuals is a less common feature in certified flight simulators , than clouds/fog/rain/snow, but sandstorms are commercially available.

depending on your needs, you can go about the modeling of underlying physics for volumetric cloud visuals in very complex ways, including modelling fluid motions, thermodynamic processes, buoyant forces, and water phase transitioning. Yet cloud visuals are "fluff", to a degree, not on the top of the list of requirements, appendix H to FAR 121 lists few requirements for visuals (the closest you get to clouds is that they mention proximity of a thunderstorm): https://www.law.cornell.edu/cfr/text/14/part-121/appendix-H

I once did some work for a system integrator who built various simulators, in the last millenium. Entire books were devoted only to 3D clouds, and complexity of visuals and simulations has increased a lot since then, is all I know :) maybe ask your training dept, or friendly sim instructors, if they have any tech specs lying around?

I can't answer how many simulators are equipped with what technology, vendors would be the best source for that information, they usually visit know their (potential) customers, and get a birds-eye view I don't have now.
here are some pointers, not sure if they help:

Clouds (http://vterrain.org/Atmosphere/Clouds/)

https://www.metavr.com/products/vrsg/IG.html#Environment

SilverLining 3D Clouds and Skies for OpenGL and DirectX (http://sundog-soft.com/sds/features/real-time-3d-clouds/)

[ZFT]

Simulators (actually FSTDs) are not certified, they are qualified.

Today, all visual systems from the 3 principle visual manufacturers, Rockwell Collins, CAE and (IMHO, the best) RSI have all the features that the OP describes and more (volcanic ash clouds and so on).

I'm not too sure just what the actual core question is?

[peekay4]

I think the op's question is more on how variable visibility (i.e., the slant range through snow, fog, rain, etc.) is modeled in simulated mixed conditions.

[deptrai]

in pure 3D graphics, if you are located

here: X looking in this ----> direction, and there is an ----> Object A here <--- in front of and covering ----> object B here <---, then Object B will not be rendered by the graphics engine, just like it is hidden and not visible in the real world. A general "Visual range", or "slant visual range" per se isn't modelled, just which exact objects are unobstructed in your line of view, or hidden behind other objects (and the 3D grid objects could be raindrops, snowflakes, smoke, fog, clouds, buildings, airplanes, bicycles, birds, blowing sand, trees, whatever, all in various levels of simplification, coarser or more detailed, in various resolutions, with various physical properties, transparencies and opacities). Add light sources to that, and you have an image of a represented physical world, limited by display system resolution, calculation depth, and the resolution of your eyes. I'm afraid I don't understand the question though.

[slast]

Thanks all for the contributions, now I understand a little bit more about how it works I will try and re-frame the question, so it's a bit more intelligible! But not tonight....

[deptrai]

There is a particular FAA required test to qualify simulators, to assess the accuracy of the visual scene at decision height during an ILS approach. Aircraft trimmed in landing configuration at 30m wheel height above ground, on glide slope, with RVR setting of 300 or 350m: "threshold lights computed to be visible must be visible, and the correct number of approach lights must be visible".

The way I understand it, there is 3d polygon model of the virtual world, including the surface/runway, with lights (not the only light sources), and there is simulated fog (or sand or smoke or anything), the fog can be patchy or non-homogenous (with a vertical variation in horizontal visibility). The fog can be simulated in various complicated ways in modern simulators (for example like this (www-igm.univ-mlv.fr/~biri/Recherche/mabiblio/GB10/GB10.pdf) ) as a particle mass which is "in between" your eyes and the threshold/approach lights (and patches of fog can even look like fog, not like a sperm whale or klingon starship as in some older simulators). The fog is a participating medium in the visuals, it affects the scene. The visual system/graphics engine knows where you are in this virtual 3D world, and it calculates what parts of the runway "behind" the fog are visible from your point of view in the cockpit, based on the simulated physical/optical properties of the fog obstructing your view, and light sources, and renders this to be displayed.

[alf5071h]

Steve, perhaps not directly relevant, but some history.
As you may recall the backdrop to European AWO was ECAC doc 17; based on these requirements the UK CAA used a linear model to determine operational minima for each aircraft type (circa 1960 / 70s). IRC the model ran in BASIC and matched straight line SVR against RVR for a first contact point against altitude. The main variables were flight-deck cut-off, RVR, and light intensity.
There may have been later versions based on RAE data in actual fog (see below*).

RAE generated and used various fog models using straight line (segmented) interpretations of SVR against RVR; these were used during extensive simulation in 1970s. RAE reports may have been by J Penwill.
A further electronic fog model was developed using a vertical scanning RVR sensor (measurements of actual RVR against altitude up to 100ft?) which predicted SVR. The predictions were validated by inflight reporting, which in my experience were very good. There were several reports in this area including approach lighting by Smith and Puffett.

N.B. The most realistic simulation ‘fog model’ associated with RAE Cat 3 research circa late 1970s consisted of a Perspex disc where segments sanded with different grades of sandpaper obscured the projection of runway lights as a function of altitude!

*Text from a post elsewhere:
The diagrams below are taken from research documents and show the first contact height and visual segment for a range of RVRs against altitude (m)
In (1), the first contact height is at the intersection of fog ‘RVR’ line and cockpit cut-off, i.e. 60M/200ft altitude in 600m RVR.
The visual segment for a particular aircraft type (cockpit cut-off) at lower altitudes, is the horizontal distance from cockpit cut-off line to the fog line, i.e. at 150 ft in 600RVR, the far point is 330m and near point is 110m; vis seg 330-110=220m. The crew would see the centreline and crossbars 2 and 3.
The threshold would be seen in this fog (600m) at an altitude of 35m / 110ft.
http://i42.tinypic.com/166kxo3.jpg

For differing fog types/cloud conditions, the lines of RVR will has a different line shape and path.
In the 600m fog shown, the first contact and ‘immediate’ visual segment (just below 200ft) would contain the centreline and crossbar 4. Just enough for Cat 1?
http://i43.tinypic.com/30u63iw.jpg

Cat 1 is generally associated with cloud breaks thus a large visual segment would be expected; however in fog, the conditions can have significant variations.
In a shallow fog (2), the threshold might be seen at 200 ft even in an RVR of 300m; hence a min RVR / approach ban.

In more mature fogs (3), nothing will be seen, even though the RVR is 600m.
http://i42.tinypic.com/xbblmd.jpg


These diagrams are from the RAE Cat 2 report ‘Manual Landings in Fog’ (Newberry).
It’s on the CD which is in the post ;)

I don’t have any direct information on snow or sand, but from limited experience in Cat2 snow, there is little variation in SVR with altitude once below cloud, which could be very low. However, in blowing snow there can by variations with altitude, but perhaps none at an altitude which includes the flight-deck or is of significance to the decision / landing.
However, again from experience, a very shallow layer of blowing snow is extremely disorienting particularly where the aircraft is pointing into wind and thus appears to drift ‘up-wind’; furthermore, the illusion changes during drift-alignment. Add to that a small touch of wing-down, - then an autoland is recommended even in good Cat 2 RVRs.
I suspect the sand is very similar.

AFAIR (late 1990s) training simulators modelled visibility in similar way to the RAE segments.
The practical difficulty is choosing an appropriate training model particularly where there are many choices.
There are few operations which allow RVR minima to be limiting is a decision (80% success rate, etc), particularly in Cat 3 fog where the fog is more homogeneous with decreasing RVR. The difficult areas are during fog formation (Cat 2) with layered structures (onion rings), or fog dispersal (mini cumulus) which can generate changes in SVR, but again from experience the SVR change is not significant in comparison with change of light levels particularly with the glare of strong sunlight above or runway lights below; I have never seen these effects modelled in simulators.
From the diagrams above, Cat 2 operations have the greater risk of having made a decision to ‘land’ (continue), and the decision subsequently be ‘wrong’.

[slast]

Thanks for all that stuff, very useful background for me.

To respond to deptrai's legitimate "why are you asking??" question, I guess I am trying to get a feel for whether routine training gives line pilots the impression that if they can see a few lights at DH and are not too seriously mis-aligned, it is actually OK to continue, because it's been repeatedly demonstrated to them that whenever they do, the visual sequence that unfolds DOES pretty quickly provide enough information to get onto the runway.

While that may be the case in a lot, even the majority, of real-world cases, it doesn't represent the dangerous cases where the aircraft's vertical trajectory is actually too steep, but the actual visual cues don't give enough information to detect it, in the patchy fog/blowing snow/ heavy rain etc., or indeed the fact that RVR reports don't actually tell you what you'll see at DH.

Given the commercial pressure to minimise simulator time I suspect that it isn't normally the case that instructors are expected to demonstrate these variable conditions. I seem to recall that a lot of that type of stuff was what we did when we first started getting into the Cat 2 /3 trials etc. using the fairly primitive systems that Alf refers to, but I don't imagine it's part of many recurrent or even initial qualification training syllabi today.

(As an aside, I vividly remember thinking my career was over during a conversion course exercise with an early visual system. This used a physical terrain model on a rolling carpet loop, with a monochrome TV camera moving on a gantry to represent "aircraft position". During maintenance this had somehow been left hooked up with left-to-right reversal (mirror image), so when we came back to do engine cuts on takeoff, rudder inputs caused the visual presentation to go in the opposite direction to the instruments, with very rapid visual negative feedback, and repeated ground loops .....!)

Anyway my concern is whether we are really training pilots to make the ACTUAL decision needed at DH, i.e. judge "position and rate of change of position relative to desired flight path". Or is the training regime acclimatising them to a rather benign situation which leads to over-confidence, and assumptions that if

1) the RVR is said to be at or above your minima, and
2) you are reasonably close to the vertical profile, and
3) you can see almost any lights at all,

then you'll be OK to continue down. Because that does seem to be a factor in a lot of accidents which are still going on.

[deptrai]

the FAA mandated test to qualify simulators in this scenario does seem a bit coarse. JAA is even more vague, I think. FAA asked for input to revise requirements last year, and one goal is to improve realism of visual cues, given advances in computer graphics (driven by mass-market applications such as games now), but requirements tend to end up to be specified in very benign ways, like "operators are encouraged to...".

[alf5071h]

Steve, your concerns and assumptions are well founded.
Adding to those, consider the way in which the simulation is used. For the mandated ‘check’ of a GA below DH due to loss of visual cues, many crews only experience the simulated visual scene being set to zero RVR, which a simplifies (forces) the decision. Whereas in reality the visual scene may slowly reduce, suckering a pilot to continue in reducing conditions, where the decision to discontinue the approach is complex and requires experience.
Whist the ‘misuse’ of the simulation provides the regulated check, it provides no opportunity to experience and learn of the difficulties in making a GA decision in variable visibility – training does not reflect actual operations.

Another issue is that with the advent of the JAA, many of the European operating minima were harmonised to the lowest limit allowed by the national authorities. At that time, precision approaches were ILS, where the minima were based on high accuracy beams minimising the need to manoeuvre below DH.
More recently, so called precision approaches with lesser, but acceptable approach accuracy (guidance and /or displays) have been authorised to the same limits. These approaches may expose crews to rare and possible not-trained-for conditions which require manoeuvres below DH. The visual cues required for manoeuvre differ from those for landing straight-in, both for visual segment and content of the visual scene. Some of these aspects also apply to non-precision approaches.

[slast]

Thank you Dan... your concerns and assumptions are well founded.
I was hoping to hear that "no, the sims are much better than you remember them and pilots today are routinely shown much more realistic visual segment displays" !
Combined with the discussion on current training issues in another thread, not a happy scenario. The technology is clearly much better but much of the industry is not making best use of it under cost pressure, and political pressures on the regulators mean they are pretty much impotent. Worrying...:(

[ZFT]

your concerns and assumptions are well founded.

I will disagree. I don’t know where your reference point was but even going back just a short few years, the latest sims ARE much better than you remember them and pilots today ARE routinely shown much more realistic visual scenes. In fact, I would suggest that of all the elements of FSTDs, the visual has shown the most improvement.

Visual manufacturers have been able to take full advantage of the mass market gaming industry and as a result the power and graphics capabilities now available to the (low volume) simulation market was unimaginable just a few years ago. Likewise, the home theatre led projection mass market has made cost effective 4 Mp projectors systems more common place on simulators allowing up to 210° X 45° FOV 3 channel systems with high brightness, great definition and excellent and distinct colours.

Of course, all these capabilities needs to be used and improved database modelling techniques with associated OS or real times with the ‘special effects’ have harnessed these with fantastic results.

Cost is not an impediment to upgrade to this later and far superior technology as typically they are 50% CHEAPER to purchase than the previous generation systems and the operating costs are minimal so why ‘industry’ doesn’t upgrade is somewhat baffling to me. (In our case we have upgraded every FFS to this latest technology).

Your original question referred to the low visibility operation and again, this area is much improved with far greater control of the environment being available to the instructor. However, I find myself being in full agreement with something that AirRabbit constantly states in so much that unless the instructor is totally familiar with the particular FSTD operation, typically these additional features and controls are not (never?) utilised to their full extent.

Unfortunately many TDMs still design their FSTDs for engineers to operate not instructors and quite often a pseudo type rating is required just to operate the damn devices!!

[deptrai]

In fact, I would suggest that of all the elements of FSTDs, the visual has shown the most improvement.

I fully agree. Yet in a complex scenario like the one Slast describes, even more realism can and will be technically achieved, computer graphics continue to improve at a pace that makes such improvements very affordable, as ZFT also pointed out.

Unfortunately many TDMs still design their FSTDs for engineers to operate not instructors and quite often a pseudo type rating is required just to operate the damn devices!!

You can (probably rightly) blame the manufacturers for making "the damn things" complicated to operate, but there is also a reluctance to purchase non-tangible goods like knowledge transfer and training of trainers. It's easier to justify the purchase of tangible whizzbangs, than to pay for non-tangible goods like services to help instructors make the best use of them. Some "purchasing department" person who is measured on cost-cutting will ask the supplier for more detailed line items on a bill, and suggest to cut non-tangible items.

Anyway my concern is whether we are really training pilots to make the ACTUAL decision needed at DH, i.e. judge "position and rate of change of position relative to desired flight path". Or is the training regime acclimatising them to a rather benign situation which leads to over-confidence

this is still a relevant question.

[slast]

Or is the training regime acclimatising them to a rather benign situation which leads to over-confidencethis is still a relevant question. By chance I just found a 2007 Canada TSB accident report which addresses this subject - I have posted a bit of background under the Halifax A320 accident thread but the recommendation 07-03 is "The Department of Transport mandate training for all pilots involved in Canadian air transport operations to better enable them to make landing decisions in deteriorating weather."

Alf- Disk arrived, thanks.

[alf5071h]

ZFT, the significant point is that irrespective of the improvements in technology, which have been many, the effectiveness of simulation remains with the instructor. Thus the issue is with the way the technology is used, not the quality of the technology.

[ZFT]

the effectiveness of simulation remains with the instructor. Thus the issue is with the way the technology is used, not the quality of the technology. alf5071h, whilst I 100% agree that without a competent and effective instructor the level of technology is immaterial, I would suggest that the effectiveness of simulation is based upon the technology and the way it is implemented, integrated and of course maintained with the effectiveness on how it is utilised (by the instructor) being the critical influence on the effectiveness of training.

I still remain a proponent of total quality in the 3 elements that I believe lead to the desired outcome of crews being trained to a level of competency that will ensure safe flight. The elements being the highest fidelity training tools, the most effective training programs and the most suitable instructors and examiners for training and checking. Any element less than ideal and the outcome will be compromised.

The big issue(s) as I see it is that the regulators and to a degree the major airframe OEMs and TDMs are either too conservative or just to slow to react to changes that the industry is crying out for. There’s lots of talk and has been for years with very little concrete action.

FSTD development by the TDMs (other than visuals) today is based upon efficiency and cost cutting not improvements in fidelity as the TDM market is now too competitive (and needs consolidation?) with the major airframe OEMs being the biggest benefactor with some 60% of an FSTD price going directly to them for P & D! These OEMs control the quality/fidelity of systems and aero simulation with the TDMs being integrators only. Yet today this data as supplied is often lacking with limited recourse for the operator to get resolution to even accepted training issues with known training issues often going on for literally years. (ARINC 610 implementation with B737NG FMS U10.x is a very good example).

FSTD fidelity improvements which should be regulatory driven are currently being operator driven and driven by that operators identified training needs (invariably by their instructors) but at a cost and in many cases, a high cost that has to be recovered.

Approved training programs typically are all based upon the OEMs recommendations and regulatory requirements and as has been clearly recognised by industry, the structure and content of training programs in use today is in many cases not relevant to current aircraft or operations.

Finally, the last element, the trainers and checkers. It is somewhat strange in this era of over regulation that any qualified trainer or checker can just climb into any qualified FSTD and train or check any trainee without any prior knowledge or experience of that FSTD. There needs to be some form of operators ‘license’ for SFI/TRI/TRE on each and every specific FSTD. This will encourage the operators and TDMs to standardise IOSs and will formalise instructor briefings on individual FSTD characteristics/discrepancies. (In our case we do offer such courses and briefings, not as a money making exercise but F.O.C. to ensure the instructors do get the best out of the FSTDs). It may also encourage the TDMs to listen to operators training needs more?

However, I would suggest that the regulations for FSTD standards both technical and operational already exist and just need to be enforced, not on the operator as such but onto the source of the problems. Likewise training programs relevance and instructor ‘licenses’ should be a regulatory matter and addressed accordingly.

IMHO only an effective regulator can make the necessary changes within the training industry that are both long overdue and desperately required. From our current experiences, their emphasis is more concentrated on their perceived benefits of training organisations complying with ICAO Doc 9859 v3 and not where it matters - TRAINING.

As an aside, being in the twilight of my career one tends to reminisce (a lot!!) and only recently I was doing just that sadly at a funeral of an old ex colleague’s wife and we were discussing training in the early 70s. Then, by today’s standards archaic training tools, chalk and talk and a 3 axis early digital FFS with model board visual on difficult aircraft such as B707C operating in a difficult environment yet in many ways not a lot has actually improved. From memory the courses were quite similar, the instructors were all very competent and then, time (cost) wasn’t a factor. Certainly the desired output seemed to be achieved. The big difference then was that the operator was in total control of all the elements with the simulator manufacturers building what you specified, in house training programs designed to suit your operations/environment and instructors invariably highly experienced and knowledgeable of both the aircraft and simulator being in house staff again.

I’m really not sure which was/is better!!!

[alf5071h]

ZFT, we are looking in the same direction, but many industry views place far too much emphasis on simulator fidelity (technological excellence) vice training program planning and instructors (as perhaps you might do).
All three components are required in proportion, but there is a tendency to downgrade those elements involving variability in human intervention – how good is the instructor day-in-day-out, does the training plan meet the need or are operational circumstances overtaking the plan.

The focus on technology is often reinforced by weak or less well thought-out evaluations of need, misjudging the diminishing value of fidelity against cost (time), and enthusiastic sales teams.

I agree that regulators etc are conservative and slow to react, but in this instance this could be of benefit to the industry; how much more can be spent on simulation.
A major problem facing the industry is that major accidents are poor indicators of specific or identifiable factors which could be amenable to training; accidents are increasingly systemic, the conjunctions of seemingly irrelevant issues not considered beforehand. These are increasingly difficult if not impossible to define, thus arguably unable to be simulated, trained for.
Some key drivers might be identifiable and included in training; e.g. automation surprise, but can simulation ever surprise a human to the edge of fear, or evoke similar emotions.
There remain significant limits to human performance, and even if a high level of behavioural training is achieved, consistent performance cannot be guaranteed in reality.

Is the underlying belief that bigger, ‘better’, shinier, technology, must provide better training, obscuring the need to consider if the current safety levels (although achieved by training) can be further improved by more training. There is a limiting point; are we at that point.