spannersatcx
23rd Dec 2000, 14:47
Just got this in the latest is of the <A HREF="http://www.lae.mcmail.com" TARGET="_blank">ALAE Tech Log</A> It is a letter from the head of CAA Engineering.
LETTER TO ASSOCIATION OF LICENSED AIRCRAFT ENGINEERS
FROM JIM MCKENNA
ORAL EXAMINATIONS UNDER JAR-66
I would like to thank those who contributed at the recent meeting to discuss the subject of JAR-66 oral examinations, particularly for the frank but balanced manner in which observations and views were expressed and the discussion progressed. A number of points were raised during the discussions, which have been considered further. These are discussed further below along with the rationale that such considerations require.
It is worthwhile noting at the outset that the committee did not unanimously agree to the retention of an oral examination with views being expressed for and against. A question was posed early on as to the legality of conducting an oral and the potential for challenge. It is clear that the regulatory philosophy under JAR's is different in some respects to previous UK practice. It is also a fact that some of the JAR's have different legal connotations falling under the scope of EC Regulation 3922/91. These are discussed later. As far JAR-66 is concerned, the rule and the associated IEM and AM material (reference JAR66.25) require that knowledge be demonstrated by examination but without directly prescribing the format of the examination. Chapter 23 of the JAA Administrative and Guidance Material does however provide information on procedures and examination formats.
It is clearly stated that written multiple choice and essay questions are to represent the primary means of examination. Whilst an oral examination is permitted at the discretion of the relevant JAA-NAA such exam elements should not be used as the primary means of establishing technical knowledge but can be used to meet the applicant and establish their fitness (understanding of their duties and responsibilities). The UK Government has undertaken to adopt and implement harmonised technical standards (JARS) through its obligations under the 1990 Cyprus Arrangements, which set up the Joint Aviation Authorities. As such the unilateral retention of an oral by the UK could well establish an unlevel playing field leaving the UK open to criticism about harmonisation despite there being apparent provision for such exams. It should also be remembered that every opportunity was afforded to every UK licensed engineers maintenance Organisation and representative bodies to offer comment on the proposed JAR-65 and JAR-66 requirements through the JAA process. Despite the CAA advising UK industry, through Airworthiness Notices, the opposition to the JAA proposal for the function of the oral exam was negligible.
The next issue to comment upon is the relationship of JAR-66 to maintenance certification. JAR-66 was primarily intended to be used in conjunction with JAR-145 although a JAA-NAA, such as the CAA, may elect to use it as a sole code. Taking JAR- 145 and commercial air transport the JAR-66 licence is only part of the prerequisite criteria for authorisation issue. The licence represents the underpinning attestation of knowledge and minimum experience. The authorisation focuses on individual competence which, over and above the licence, includes requirements for recency of experience, differences training, practical ability, procedural knowledge and continuation training. In many respects, this philosophy reflects many of the facets developed under BCAR A8-13 for qualification of maintenance staff.
Under that requirement, the CAA had focused the qualification process from type oral exams and licence type ratings to basic licences complemented by type training and authorisation within approved companies. In 1988 the CAA further changed the licensing requirements by introducing prerequisite training courses for certain aircraft types rather than continuing type oral examinations. In many respects this arrangement has proven to be advantageous to both industry and the CAA and has helped keep licensing costs to a minimum. It is clear however, even at our meeting, that some licensed engineers view these authorisation requirements with a degree of mistrust, believing that some companies are abusing the process and issuing authorisations against minimal training and experience. The CAA's own experiences in evaluating authorisation and training standards associated with the transfer of 'protected rights' under JAR-66 tends to support this perception of abuse to some extent. There may clearly be an issue here demanding closer supervision of the activity through JAR-145 monitoring. Notwithstanding this the philosophy of BAR A8-13 has proven to be sound as evidenced by the many hundreds of flights that operate successfully everyday.
(so that makes it right then does it, spannersatcx)
There appeared to be acceptance across the committee that the JAR-145 Organisation ultimately has the responsibility for determining the competence of the 'individual. It cannot be denied that the JAR-145 Organisation is ultimately responsible for the safety of the end product and the oversight of the individual within the process. The CAA advocated this was an ongoing issue requiring, to an extent, continuous assessment of individual performance and one, which could not be properly established by a point in time oral examination. The CAA further suggested that an assessment within the workplace, under real conditions and pressures was the most effective way of determining competence. I do not recall any dissent within the committee on this view.
Some of the committee suggested the oral offered an opportunity to meet CAA staff since surveyors were rarely to be seen on the shop floor. If there is a lack of CAA presence as part of our industry oversight an oral examination at licence issue is not the answer. The CAA must reconsider its monitoring policy to resolve this. Another committee member stated that the oral allowed the CAA the opportunity to assess the individual's suitability, the Organisation that he represented further advocating the introduction of psychometric testing. Both these concentrate on a point in time oral exam being used to assess how an individual will behave over their working career. It is not realistic to believe that such assessments have long-term value.
Subtle personality disorders or future false intent cannot readily be detected by a short exam. Such traits are only measurable or detectable through monitoring behaviour over time. The weakness of the oral at present is that it cannot detect a tendency to mental instability nor whether the individual will actually adhere to the requirements and above good maintenance practice. Claims that the oral is essential to the safe maintenance of aircraft are arguably wildly exaggerated since behind every maintenance error a licensed engineer has failed either directly or by inadequate supervision of staff within his control.
(so it's all our fault is it, forget human factors then, spannersatcx)
There is clearly a need for doing something in the case of engineers wishing to work outside of JAR- 145. Aircraft most likely to fall into the non-commercial air transport areas are general aviation or light aircraft. Within the proposals to extend JAR-66 to cover light aircraft the majority of aircraft will not require a formal type-training course as a prerequisite to licence issue. At present it appears that a technical oral exam is acceptable as a means of determining knowledge. Since a JAR-145 Organisation is not involved a further element to the technical oral to cover fitness issues and the knowledge of duties and responsibilities under AN Article 12 and Airworthiness Notices could be developed.
The CAA has given long and careful consideration to this matter and has weighed up the various comments made. It is clear that, despite the general desire of engineers to retain the oral, continuing this practice just because engineers have previously had to suffer it is insufficient justification for retention. JAR- 145 and the previous philosophy behind BAR A8-13 firmly places the responsibility for determining individual competence within the maintenance Organisation. If licensed engineers suspect that companies are not discharging this responsibility properly the CAA should consider 'increased monitoring of the process. Again, using the oral exam at basic licence issue to cater for organisational shortfalls is not justifiable. Where there is a need to establish confidence that the individual knows what they are responsible for i.e. outside of JAR-145, the type oral examination can be adjusted to fit.
On balance therefore there is insufficient justification in our opinion to continue the oral examination at basic licence issue.
[This message has been edited by spannersatcx (edited 23 December 2000).]
LETTER TO ASSOCIATION OF LICENSED AIRCRAFT ENGINEERS
FROM JIM MCKENNA
ORAL EXAMINATIONS UNDER JAR-66
I would like to thank those who contributed at the recent meeting to discuss the subject of JAR-66 oral examinations, particularly for the frank but balanced manner in which observations and views were expressed and the discussion progressed. A number of points were raised during the discussions, which have been considered further. These are discussed further below along with the rationale that such considerations require.
It is worthwhile noting at the outset that the committee did not unanimously agree to the retention of an oral examination with views being expressed for and against. A question was posed early on as to the legality of conducting an oral and the potential for challenge. It is clear that the regulatory philosophy under JAR's is different in some respects to previous UK practice. It is also a fact that some of the JAR's have different legal connotations falling under the scope of EC Regulation 3922/91. These are discussed later. As far JAR-66 is concerned, the rule and the associated IEM and AM material (reference JAR66.25) require that knowledge be demonstrated by examination but without directly prescribing the format of the examination. Chapter 23 of the JAA Administrative and Guidance Material does however provide information on procedures and examination formats.
It is clearly stated that written multiple choice and essay questions are to represent the primary means of examination. Whilst an oral examination is permitted at the discretion of the relevant JAA-NAA such exam elements should not be used as the primary means of establishing technical knowledge but can be used to meet the applicant and establish their fitness (understanding of their duties and responsibilities). The UK Government has undertaken to adopt and implement harmonised technical standards (JARS) through its obligations under the 1990 Cyprus Arrangements, which set up the Joint Aviation Authorities. As such the unilateral retention of an oral by the UK could well establish an unlevel playing field leaving the UK open to criticism about harmonisation despite there being apparent provision for such exams. It should also be remembered that every opportunity was afforded to every UK licensed engineers maintenance Organisation and representative bodies to offer comment on the proposed JAR-65 and JAR-66 requirements through the JAA process. Despite the CAA advising UK industry, through Airworthiness Notices, the opposition to the JAA proposal for the function of the oral exam was negligible.
The next issue to comment upon is the relationship of JAR-66 to maintenance certification. JAR-66 was primarily intended to be used in conjunction with JAR-145 although a JAA-NAA, such as the CAA, may elect to use it as a sole code. Taking JAR- 145 and commercial air transport the JAR-66 licence is only part of the prerequisite criteria for authorisation issue. The licence represents the underpinning attestation of knowledge and minimum experience. The authorisation focuses on individual competence which, over and above the licence, includes requirements for recency of experience, differences training, practical ability, procedural knowledge and continuation training. In many respects, this philosophy reflects many of the facets developed under BCAR A8-13 for qualification of maintenance staff.
Under that requirement, the CAA had focused the qualification process from type oral exams and licence type ratings to basic licences complemented by type training and authorisation within approved companies. In 1988 the CAA further changed the licensing requirements by introducing prerequisite training courses for certain aircraft types rather than continuing type oral examinations. In many respects this arrangement has proven to be advantageous to both industry and the CAA and has helped keep licensing costs to a minimum. It is clear however, even at our meeting, that some licensed engineers view these authorisation requirements with a degree of mistrust, believing that some companies are abusing the process and issuing authorisations against minimal training and experience. The CAA's own experiences in evaluating authorisation and training standards associated with the transfer of 'protected rights' under JAR-66 tends to support this perception of abuse to some extent. There may clearly be an issue here demanding closer supervision of the activity through JAR-145 monitoring. Notwithstanding this the philosophy of BAR A8-13 has proven to be sound as evidenced by the many hundreds of flights that operate successfully everyday.
(so that makes it right then does it, spannersatcx)
There appeared to be acceptance across the committee that the JAR-145 Organisation ultimately has the responsibility for determining the competence of the 'individual. It cannot be denied that the JAR-145 Organisation is ultimately responsible for the safety of the end product and the oversight of the individual within the process. The CAA advocated this was an ongoing issue requiring, to an extent, continuous assessment of individual performance and one, which could not be properly established by a point in time oral examination. The CAA further suggested that an assessment within the workplace, under real conditions and pressures was the most effective way of determining competence. I do not recall any dissent within the committee on this view.
Some of the committee suggested the oral offered an opportunity to meet CAA staff since surveyors were rarely to be seen on the shop floor. If there is a lack of CAA presence as part of our industry oversight an oral examination at licence issue is not the answer. The CAA must reconsider its monitoring policy to resolve this. Another committee member stated that the oral allowed the CAA the opportunity to assess the individual's suitability, the Organisation that he represented further advocating the introduction of psychometric testing. Both these concentrate on a point in time oral exam being used to assess how an individual will behave over their working career. It is not realistic to believe that such assessments have long-term value.
Subtle personality disorders or future false intent cannot readily be detected by a short exam. Such traits are only measurable or detectable through monitoring behaviour over time. The weakness of the oral at present is that it cannot detect a tendency to mental instability nor whether the individual will actually adhere to the requirements and above good maintenance practice. Claims that the oral is essential to the safe maintenance of aircraft are arguably wildly exaggerated since behind every maintenance error a licensed engineer has failed either directly or by inadequate supervision of staff within his control.
(so it's all our fault is it, forget human factors then, spannersatcx)
There is clearly a need for doing something in the case of engineers wishing to work outside of JAR- 145. Aircraft most likely to fall into the non-commercial air transport areas are general aviation or light aircraft. Within the proposals to extend JAR-66 to cover light aircraft the majority of aircraft will not require a formal type-training course as a prerequisite to licence issue. At present it appears that a technical oral exam is acceptable as a means of determining knowledge. Since a JAR-145 Organisation is not involved a further element to the technical oral to cover fitness issues and the knowledge of duties and responsibilities under AN Article 12 and Airworthiness Notices could be developed.
The CAA has given long and careful consideration to this matter and has weighed up the various comments made. It is clear that, despite the general desire of engineers to retain the oral, continuing this practice just because engineers have previously had to suffer it is insufficient justification for retention. JAR- 145 and the previous philosophy behind BAR A8-13 firmly places the responsibility for determining individual competence within the maintenance Organisation. If licensed engineers suspect that companies are not discharging this responsibility properly the CAA should consider 'increased monitoring of the process. Again, using the oral exam at basic licence issue to cater for organisational shortfalls is not justifiable. Where there is a need to establish confidence that the individual knows what they are responsible for i.e. outside of JAR-145, the type oral examination can be adjusted to fit.
On balance therefore there is insufficient justification in our opinion to continue the oral examination at basic licence issue.
[This message has been edited by spannersatcx (edited 23 December 2000).]