CRM Assesment
Join Date: Feb 2005
Location: On the green bit near the blue wobbly stuff
Posts: 674
Likes: 0
Received 1 Like
on
1 Post
Picklepilot,
If they are capable and experienced pilots.... Yes, I think CRM is EXACTLY what they will be using to deal with a problem in the cockpit! I would like to imagine that they would have extremely good CRM. I.E: leadership, management, decision-making, situational awareness, communication skills, cooperation etc. Why would they not use these CRM skills because they have 10000 hours ?????? However many hours we have, only the most arrogant will claim that they have nothing left to learn or refine. Believe it or not, 10,000 hour pilots are subject to human error too.
If they are capable and experienced pilots.... Yes, I think CRM is EXACTLY what they will be using to deal with a problem in the cockpit! I would like to imagine that they would have extremely good CRM. I.E: leadership, management, decision-making, situational awareness, communication skills, cooperation etc. Why would they not use these CRM skills because they have 10000 hours ?????? However many hours we have, only the most arrogant will claim that they have nothing left to learn or refine. Believe it or not, 10,000 hour pilots are subject to human error too.
Join Date: Oct 2005
Location: USA
Posts: 3,218
Likes: 0
Received 0 Likes
on
0 Posts
Picklepilot, known formerly as ssg the banned poster, makes frequent reference to his vast simulator experience. Microsoft simulator, most likely. A troll who offers nothing, he will soon be banned like all his other names.
CRM isn't an airline concept, bright one. It's a crew concept, it's a single pilot concept, and it's applicable in every type of cockpit environment there is. CRM includes those outside the cockpit as well as in, it brings in the controller, it brings in everybody. It's heavily emphasized in the corporate cockpit, the fracitonal cockpit, the military cockpit; it's the basis of doing business.
Again, clearly you are without experience or knowledge in this area, and again, clearly you embarass yourself. You have an agenda to attack airlines, but know nothing about them, or about any other segment of the industry. You've tried to pass yourself off as a charter pilot, an airline pilot, a helicopter pilot, a corporate pilot...and you keep posting this ridiculous stupid statements. Can you not help yourself?
You know if the airlines hired more experienced pilots vs marshmellows, this CRM stuff wouldn't be so neccessary.. CRM is a result of strong capts screwing up and the FO watches the capt fly it to the scene of the accident. Hiring better pilots precludes this whole issue of needing CRM Do you really think two 10000 hour pilots are going to need CRM when trying to figure out what's wrong in the cockpit?
Again, clearly you are without experience or knowledge in this area, and again, clearly you embarass yourself. You have an agenda to attack airlines, but know nothing about them, or about any other segment of the industry. You've tried to pass yourself off as a charter pilot, an airline pilot, a helicopter pilot, a corporate pilot...and you keep posting this ridiculous stupid statements. Can you not help yourself?
Join Date: Feb 2005
Location: On the green bit near the blue wobbly stuff
Posts: 674
Likes: 0
Received 1 Like
on
1 Post
Whoops, didnt realize that the gherkin had been previously identified as a git - was giving him the benefit of the doubt. Will try not to take the bait next time!
Join Date: Jul 2002
Location: The Frozen North
Posts: 88
Likes: 0
Received 0 Likes
on
0 Posts
While Canadian Rides do not have a CRM pass/fail assessment as yet, a recent change focuses on CRM attributes duinr the ride debrief.
Transport Canada has a DVD they released on assessing CRM, and the format for ride debriefing is the Nasa/UTexas format.
http://homepage.psy.utexas.edu/homep.../HelmreichLAB/
Nasa's paper on ride debriefing is here:
http://human-factors.arc.nasa.gov/fl...tions/IJAP.pdf
Nasa's rather impressive site for publications is here:
http://human-factors.arc.nasa.gov/fl...lications.html
Transport Canada has a DVD they released on assessing CRM, and the format for ride debriefing is the Nasa/UTexas format.
http://homepage.psy.utexas.edu/homep.../HelmreichLAB/
Nasa's paper on ride debriefing is here:
http://human-factors.arc.nasa.gov/fl...tions/IJAP.pdf
Nasa's rather impressive site for publications is here:
http://human-factors.arc.nasa.gov/fl...lications.html
Thought for the day ...
Would it be simpler to pick crew who are objective and not easily offended, rather than picking crew members who are over-elaborately trained to avoid giving offence?
If I need to learn something I want to be told about it, not have it hinted at and hidden in a long list of things I didn't do wrong!
OK - now I'll go back to my SEP.
If I need to learn something I want to be told about it, not have it hinted at and hidden in a long list of things I didn't do wrong!
OK - now I'll go back to my SEP.
Not quite a DOH
situation Non PC Plod. So how do you assess people for selection to make sure they are not easily offended?? Swear at them in the simulator constantly? Word would soon get out that your company has a test for 'not being offended' at which point you could tell prospective employees in the sim (naughty comments deleted with a wry smile - JT), and they would turn and smile and say 'Roger, are you ready for the approach brief Captain?'
It is commonly accepted with CRM assessment that on any given sortie, one cannot test another on all of the behavioural markers. You can only assess those markers that arise during the conduct of the trip.
So back to you. DOH
situation Non PC Plod. So how do you assess people for selection to make sure they are not easily offended?? Swear at them in the simulator constantly? Word would soon get out that your company has a test for 'not being offended' at which point you could tell prospective employees in the sim (naughty comments deleted with a wry smile - JT), and they would turn and smile and say 'Roger, are you ready for the approach brief Captain?'It is commonly accepted with CRM assessment that on any given sortie, one cannot test another on all of the behavioural markers. You can only assess those markers that arise during the conduct of the trip.
So back to you. DOH
Join Date: Feb 2005
Location: On the green bit near the blue wobbly stuff
Posts: 674
Likes: 0
Received 1 Like
on
1 Post
Roger,
I wasnt suggesting any Homer Simpsonisms for yourself, it was a poor attempt at incisive wit on a bored afternoon.
The serious point is, if non-technical skills are as critical to the safe outcome of a sortie as are technical skills (a fact borne out by LOSA data), then we should be looking at them during selection, line training, recurrent training, and indeed, all stages of the training process. Of course, you cant prove a negative, so if someone does not throw a wobbly on one occasion in the simulator, you cannot de facto say that they are necessarily an all-round good egg. But, if we maintain good training records, hopefully a pattern will emerge if someone has a tendency, say, to rush their checks and miss items, just as if they have a tendency always to be too high on the approach. In this way we can weed out the wrong candidates during the training process IF they display poor CRM skills, just as if they displayed poor handling skills. Of course you cant artificially test all aspects of someone's character, but if you put a crew under a bit of pressure, you can see if they are capable of working together effectively as a team, whether or not they like each other, are offended, etc etc. You can see their behaviour, and make an objective assessment about it.
I wasnt suggesting any Homer Simpsonisms for yourself, it was a poor attempt at incisive wit on a bored afternoon.
The serious point is, if non-technical skills are as critical to the safe outcome of a sortie as are technical skills (a fact borne out by LOSA data), then we should be looking at them during selection, line training, recurrent training, and indeed, all stages of the training process. Of course, you cant prove a negative, so if someone does not throw a wobbly on one occasion in the simulator, you cannot de facto say that they are necessarily an all-round good egg. But, if we maintain good training records, hopefully a pattern will emerge if someone has a tendency, say, to rush their checks and miss items, just as if they have a tendency always to be too high on the approach. In this way we can weed out the wrong candidates during the training process IF they display poor CRM skills, just as if they displayed poor handling skills. Of course you cant artificially test all aspects of someone's character, but if you put a crew under a bit of pressure, you can see if they are capable of working together effectively as a team, whether or not they like each other, are offended, etc etc. You can see their behaviour, and make an objective assessment about it.
The serious point …
Non PC Plod Re: … if non-technical skills are as critical to the safe outcome of a sortie as are technical skills (a fact borne out by LOSA data)…
One of the problems in aviation is that there are few ‘real’ facts, thus it would be interesting to see the LOSA data which you suggest supports this.
I would agree that NTS are very important and should be assessed (but never scored).
Whilst the ‘social’ NTS subjects might be judged on resultant behaviour – (normal, everyday aspects of human interaction), the cognitive subjects (situation awareness, decision making) are much more difficult to assess. As with LOSA, a weakness in the assessment is that the assessor cannot know what the subject is actual thinking – the aspects of knowledge, understanding, bias, or belief that s/he applies to the situation at the time of gaining situation awareness or making decisions.
With NTS ground based assessment, there is opportunity when debriefing to explore differing viewpoints, but this is always after the fact – in hindsight. Often the subject will recall a thought process matching the outcome, as opposed to what actually was considered at the time; the subject may even be unaware of the aspects which affected the thinking (the cognitive NTS).
On the other hand, LOSA does not involve any debrief and thus any conclusion is based only that of the observer and not necessarily what the subject perceived or deduced in the thinking process. Therefore LOSA data more often fits the facts after the event (outcome as opposed to process), which in identifying errors, is always the case, i.e pilots do not intend to commit an error (excluding violations), but the outcome (in LOSA) is judged as an error, without correlating the thinking that led to it. This masks any thinking skills, which could have been very good - the subject just used the wrong data.
This might be what you describe as ‘proving a negative’ or the subject ‘throwing a wobbly’; with this, I agree. However, I suggest that LOSA cannot prove that NTS are critical to a safe out come – they probably are, but still have to be proven by some other means.
With the use of trained assessors, and with knowledge of the problems above, then simulator training (LOFT) using controlled scenarios, stop and assess capability, and opportunity for instant debrief or repetition, enables a reasonable fair ‘outcome’ based assessment.
However, outcome assessment in line flying (route checks) is more difficult due to the lack of these controls. Also, it does not guard against both crew members simultaneously misperceiving a situation or suffering groupthink to arrive at an erroneous conclusion, e.g. weather is not as bad as indicated, or a minor organizational problem is discarded by a local work-around.
As Roger states “You can only assess those markers that arise during the conduct of the trip.”
One of the problems in aviation is that there are few ‘real’ facts, thus it would be interesting to see the LOSA data which you suggest supports this.
I would agree that NTS are very important and should be assessed (but never scored).
Whilst the ‘social’ NTS subjects might be judged on resultant behaviour – (normal, everyday aspects of human interaction), the cognitive subjects (situation awareness, decision making) are much more difficult to assess. As with LOSA, a weakness in the assessment is that the assessor cannot know what the subject is actual thinking – the aspects of knowledge, understanding, bias, or belief that s/he applies to the situation at the time of gaining situation awareness or making decisions.
With NTS ground based assessment, there is opportunity when debriefing to explore differing viewpoints, but this is always after the fact – in hindsight. Often the subject will recall a thought process matching the outcome, as opposed to what actually was considered at the time; the subject may even be unaware of the aspects which affected the thinking (the cognitive NTS).
On the other hand, LOSA does not involve any debrief and thus any conclusion is based only that of the observer and not necessarily what the subject perceived or deduced in the thinking process. Therefore LOSA data more often fits the facts after the event (outcome as opposed to process), which in identifying errors, is always the case, i.e pilots do not intend to commit an error (excluding violations), but the outcome (in LOSA) is judged as an error, without correlating the thinking that led to it. This masks any thinking skills, which could have been very good - the subject just used the wrong data.
This might be what you describe as ‘proving a negative’ or the subject ‘throwing a wobbly’; with this, I agree. However, I suggest that LOSA cannot prove that NTS are critical to a safe out come – they probably are, but still have to be proven by some other means.
With the use of trained assessors, and with knowledge of the problems above, then simulator training (LOFT) using controlled scenarios, stop and assess capability, and opportunity for instant debrief or repetition, enables a reasonable fair ‘outcome’ based assessment.
However, outcome assessment in line flying (route checks) is more difficult due to the lack of these controls. Also, it does not guard against both crew members simultaneously misperceiving a situation or suffering groupthink to arrive at an erroneous conclusion, e.g. weather is not as bad as indicated, or a minor organizational problem is discarded by a local work-around.
As Roger states “You can only assess those markers that arise during the conduct of the trip.”
Join Date: Oct 2003
Location: London
Posts: 258
Likes: 0
Received 0 Likes
on
0 Posts
CRM is not in isolation a pass/fail subject as far as the caa are concerned. If the flight crew do fail in one area of there LPC ie single engine go around and it is believed that poor crm brought this about then they can fail the LPC and the fail can be attributed to CRM. Ie if you had two pilots who got in the sim, didn't say a word to each other apart from sop's and were seen to pass all the usual technical aspect of an LPC then they could not be failed on CRM (that's the easiest way of thinking about it).
Join Date: Feb 2005
Location: On the green bit near the blue wobbly stuff
Posts: 674
Likes: 0
Received 1 Like
on
1 Post
Alf,
I agree- its very difficult to "prove" that NTS are critical to a safe outcome, but there is plenty of evidence to support the proposition. Every accident or incident that is put down to pilot error adds weight to this idea.
LOSA data is of course subjective, but looking at the papers produced by Helmreich et al (University of Texas), I think anyone can see that there is a lot of useful information available provided the observers are sufficiently well-trained and capable. If nothing else, it can help to identify trends if there is a history of, for example, unstabilised approaches using a particular procedure at a particular airport. There may be a plethora of different reasons why pilots have got the aircraft into this situation, but unless someone has flagged up the fact that there is a recurrent issue not related to aircraft serviceability, then it will not get addressed until it is too late.
As Mesh says, the CAA require technical consequences before NTS causes a failure on a line check. This does not, of course, prevent a comment or exploration of CRM issues in a debrief even if the line check was passed.
There is always a problem with debriefing an event, and having your episodic memory massage the facts. But, at least a well-facilitated debrief (particularly if video debriefing stations are available), will allow you to re-visit the events and allow you to formulate better ways of coping with particular situations.
There is always something to learn - there is always someone who has had an experience that you have no come across yet.
I agree- its very difficult to "prove" that NTS are critical to a safe outcome, but there is plenty of evidence to support the proposition. Every accident or incident that is put down to pilot error adds weight to this idea.
LOSA data is of course subjective, but looking at the papers produced by Helmreich et al (University of Texas), I think anyone can see that there is a lot of useful information available provided the observers are sufficiently well-trained and capable. If nothing else, it can help to identify trends if there is a history of, for example, unstabilised approaches using a particular procedure at a particular airport. There may be a plethora of different reasons why pilots have got the aircraft into this situation, but unless someone has flagged up the fact that there is a recurrent issue not related to aircraft serviceability, then it will not get addressed until it is too late.
As Mesh says, the CAA require technical consequences before NTS causes a failure on a line check. This does not, of course, prevent a comment or exploration of CRM issues in a debrief even if the line check was passed.
There is always a problem with debriefing an event, and having your episodic memory massage the facts. But, at least a well-facilitated debrief (particularly if video debriefing stations are available), will allow you to re-visit the events and allow you to formulate better ways of coping with particular situations.
There is always something to learn - there is always someone who has had an experience that you have no come across yet.
Non-PC Plod whether provable or not, we agree that NTS will aid a safe outcome. However our opinions diverge on the evidence of pilot error from incidents, accidents, or LOSA.
Many accident reports (perhaps the more older ones) conclude pilot error, or CRM failure as causal factors, but these terms only state the outcome and not the reason for it (the old way of thinking about human error – Dekker).
Many of the errors reported by LOSA are outcomes; they do not uncover the underlying reasons for the behaviour.
Your example of unstabilised approaches from LOSA is a record of what happened; I have yet to see reasons why they happened. In one instance crews might be overly influenced by organisation pressure for on time arrivals or ATC requests (misappropriated goal), which could be considered NTS items. Alternatively, the approach procedure, aircraft configuration, weight, wind, rules / procedures may inevitably result in a fast approach, which being outside of the pilots control should not be a NTS issue – it’s a system / organisational weakness that require fixing. [You could assess the ability to identify the reasons for the fast approach and the attitude towards submitting a safety report – but that might be unfair].
I suggest that the majority of LOSA reports involve the latter situation, a system or organisational culture issue which result in violation. This is not to say that there are many occurrences of ‘press-on-it is’ (failure of cognition), but without discussion how can these be identified? Trained LOSA observers can help, but they are equally subject to bias and misperception. Is the inability to differentiate the unprovable aspect of cognitive NTS and a major weakness of LOSA?
As stated previously, NTS assessment has the advantage of discussion and to some degree control of perceptual or systems issues in ground training, but not so in airborne assessment.
Thus in-flight, if a fast approach is deemed a technical consequence, and that the underlying reason is not identified (differing opinions), then is this a NTS failure?
Not in my book; it’s a ‘no test’, there is insufficient information on which to make an assessment, but plenty for both parties to learn from.
As de Bono states (many books) there are positive, negative and ‘interesting’ aspects in most situations; I think that the majority of interesting aspects involve cognitive NTS.
Many accident reports (perhaps the more older ones) conclude pilot error, or CRM failure as causal factors, but these terms only state the outcome and not the reason for it (the old way of thinking about human error – Dekker).
Many of the errors reported by LOSA are outcomes; they do not uncover the underlying reasons for the behaviour.
Your example of unstabilised approaches from LOSA is a record of what happened; I have yet to see reasons why they happened. In one instance crews might be overly influenced by organisation pressure for on time arrivals or ATC requests (misappropriated goal), which could be considered NTS items. Alternatively, the approach procedure, aircraft configuration, weight, wind, rules / procedures may inevitably result in a fast approach, which being outside of the pilots control should not be a NTS issue – it’s a system / organisational weakness that require fixing. [You could assess the ability to identify the reasons for the fast approach and the attitude towards submitting a safety report – but that might be unfair].
I suggest that the majority of LOSA reports involve the latter situation, a system or organisational culture issue which result in violation. This is not to say that there are many occurrences of ‘press-on-it is’ (failure of cognition), but without discussion how can these be identified? Trained LOSA observers can help, but they are equally subject to bias and misperception. Is the inability to differentiate the unprovable aspect of cognitive NTS and a major weakness of LOSA?
As stated previously, NTS assessment has the advantage of discussion and to some degree control of perceptual or systems issues in ground training, but not so in airborne assessment.
Thus in-flight, if a fast approach is deemed a technical consequence, and that the underlying reason is not identified (differing opinions), then is this a NTS failure?
Not in my book; it’s a ‘no test’, there is insufficient information on which to make an assessment, but plenty for both parties to learn from.
As de Bono states (many books) there are positive, negative and ‘interesting’ aspects in most situations; I think that the majority of interesting aspects involve cognitive NTS.
Join Date: Feb 2005
Location: On the green bit near the blue wobbly stuff
Posts: 674
Likes: 0
Received 1 Like
on
1 Post
Alf,
Given that we agree that good NTS are a positive thing, and I think we agree that the best environment to examine both our own and other people's NTS are in the controlled environment of the simulator.
But - that is surely not to say that accident investigations and LOSA are of no value because they record outcomes rather than underlying reasons.
Clearly not in all, but in many cases there is sufficient evidence to infer underlying reasons (whether these are down to NTS or wider-CRM issues, such as management, supervision, regulation etc) If this can spur us to re-examine the way we do things, it can surely only help to improve the safety of our operations in the future?
Given that we agree that good NTS are a positive thing, and I think we agree that the best environment to examine both our own and other people's NTS are in the controlled environment of the simulator.
But - that is surely not to say that accident investigations and LOSA are of no value because they record outcomes rather than underlying reasons.
Clearly not in all, but in many cases there is sufficient evidence to infer underlying reasons (whether these are down to NTS or wider-CRM issues, such as management, supervision, regulation etc) If this can spur us to re-examine the way we do things, it can surely only help to improve the safety of our operations in the future?
“ … not to say that accident investigations and LOSA are of no value because they record outcomes rather than underlying reasons”.
I agree, but accident reports must be used with care as these are critically dependent on the interpretation of the instructor/reader – the inference.
Many accident reports adhere strictly to the ICAO guidance in only presenting ‘factual’ evidence, thus many of the NTS aspects, being supposition, are avoided, or glibly labeled ‘pilot error’ or CRM. This is an opportunity for poor association and reduced training value. As stated previously there are few ‘facts’ in aviation and where given, often involve hindsight or other biases – in this instance, originating at the point of use.
However, recently some accident investigations have attempted to address the human issues. In some reports, ASTB in Australia have presented human factors in a speculative manner, with arguments for and against, and concluding on the balance of probabilities. This enables a reader to consider all of the possible contributions in the context of the accident, and although not ‘provable’ as fact, are of more value than just stating the outcome.
Other investigators have also attempted to grasp the human factors issues, most very successfully, and usually with informed or speculative discussion of the cognitive issues. These approaches will benefit NTS training and provide assessors with a wider baseline of context based behaviors – the real world.
There are fewer similarities with LOSA; the value of the report depends on the observer (incident report), but to date, the reporting method does not address the ‘factors’ or even speculate on NTS. Thus, I remain skeptical about use of LOSA in this area.
However, consider if we encouraged pilots to conduct their own (personal) LOSA during normal operations (threat and error management), this might identify some unseen threats, and, with honest self reflective assessment, pilots could improve their NTS – greater opportunity for practice.
For interest:-
What Can You Learn from Accident Reports? Page 7.
Also: TRIS Online: The Effect of Accident Report Formats on Invulnerability and Hindsight Bias
Human Factors in the Investigation of Accidents and Incidents.
Why System Safety Professionals Should Read Accident Reports.
I agree, but accident reports must be used with care as these are critically dependent on the interpretation of the instructor/reader – the inference.
Many accident reports adhere strictly to the ICAO guidance in only presenting ‘factual’ evidence, thus many of the NTS aspects, being supposition, are avoided, or glibly labeled ‘pilot error’ or CRM. This is an opportunity for poor association and reduced training value. As stated previously there are few ‘facts’ in aviation and where given, often involve hindsight or other biases – in this instance, originating at the point of use.
However, recently some accident investigations have attempted to address the human issues. In some reports, ASTB in Australia have presented human factors in a speculative manner, with arguments for and against, and concluding on the balance of probabilities. This enables a reader to consider all of the possible contributions in the context of the accident, and although not ‘provable’ as fact, are of more value than just stating the outcome.
Other investigators have also attempted to grasp the human factors issues, most very successfully, and usually with informed or speculative discussion of the cognitive issues. These approaches will benefit NTS training and provide assessors with a wider baseline of context based behaviors – the real world.
There are fewer similarities with LOSA; the value of the report depends on the observer (incident report), but to date, the reporting method does not address the ‘factors’ or even speculate on NTS. Thus, I remain skeptical about use of LOSA in this area.
However, consider if we encouraged pilots to conduct their own (personal) LOSA during normal operations (threat and error management), this might identify some unseen threats, and, with honest self reflective assessment, pilots could improve their NTS – greater opportunity for practice.
For interest:-
What Can You Learn from Accident Reports? Page 7.
Also: TRIS Online: The Effect of Accident Report Formats on Invulnerability and Hindsight Bias
Human Factors in the Investigation of Accidents and Incidents.
Why System Safety Professionals Should Read Accident Reports.
Join Date: Feb 2005
Location: On the green bit near the blue wobbly stuff
Posts: 674
Likes: 0
Received 1 Like
on
1 Post
We are in agreement on that one - I wonder how often crews take the time for a debrief if they are not in a formal training environment. Is it the norm for everyone to take a couple of minutes to go through the flight and discuss whether there was anything they could have done better......or do they crack the paperwork and get off home?
