Dick,
If I may take a few of your comments in turn:
Quote: "First, in the fields of writing objectives, validating the question bank and weeding out ambiguities, I was there, for MET, and I think that we did a generally sound job"
This illustrates my point regard the lack of a proper training needs analysis. When setting about creating a set of objectives it is tempting to get together a group of interested parties and have a good old brain storming session. This might well generate a set of objectives with which most of the participants agree, but it is not really a very scientific approach. Nor is it a proper training needs analysis. Using this method invariably results in some irrelevant material being included and/or some essential material being omitted. Teaching too much is simply wasteful, but missing out essential material can be positively dangerous. A good many people have managed to kill themselves (and others in their vicinity) through not having the correct skills, knowledge and attitudes. My point regarding the validity of the objectives is simply that without a proper analysis we are unlikely to identify truly valid learning objectives.
Quote: "As we go through the exams we, the FTOs, continually ask for revision of doubtful questions. Again, the CAA is generally helpful and flexible."
I agree. Nothing in any of my posts has indicated that the CAA is anything other than (generally) helpful and flexible. But the need to continuously go back to them to query problem questions is a direct result of the failures of the initial analysis and question writing processes. If the system is so good, why did so many students get caught out last year by the sudden introduction of lots of probing questions on tacho systems. Did Bristol's notes already cover the subject to the required extent, or did you (like all the other FTOs) quickly revise them. Your notes have a reputation for being excellent, so if yours had to be improved, what does that mean for students in less excellent schools?
Quote: "Second, in dealing with specific questions, IMHO, you have to be very careful".
I agree, but SuperTed asked about the validity of the exam questions. If we do not deal with specific examples, we will get nowhere. Students complain that questions are unfair….CAA says they are not….. End of discussion.
Your discussion of Vx and Vy is excellent, but I feel we need to be careful about definitions. You are correct in saying that if you have infinite power it is aerodynamics that will eventually limit your altitude. But absolute ceiling, aerodynamic ceiling and maximum operating altitude are all different things. The syllabus requires students to have a clear understanding of each, and to be able to distinguish between the three. Students reading a couple of your comments might be confused.
I am sorry if you feel that my last post was simply intended to demonstrate my knowledge of the subject. This was not my intention. A question was asked and I responded to it. Nothing in my comments about Vx and Vy were beyond the level required by the students. My description of the training needs analysis process was intended only to provide a context for my assessment that the learning objectives probably aren't valid.
What I actaully said in the much earlier post to which you refered was "we must ensure our answers are pitched at the correct level to ensure they are of use to the recipients". That isn't quite the same as saying "don't demonstrate your knolwedge".
TINSTAAFL,
I agree with everything you have said about how the process of question/exam validation SHOULD work. But unless the original learning objectives are based on a proper training needs analysis, any validation process can do no more than confirm that both the examiners' and the FTOs are employing the same interpretation of the objectives. To take an extreme example, we might teach the students lots of interesting things about all the valve and switches in their hydraulic systems, The teaching might be excellently designed an executed and the questions perfectly constructed. But unless we tell them to put the rear down before landing ……
One of the ways in which the validation process can go wrong is illustrated in a current string in the tech log. A member asked for advice concerning the number of gimbals, axes of motion and freedoms of motion of various gyroscopes. After a couple of days I noticed that no responses had been made, so I consulted a reference (The RAF's AP 3456) and posted a response, together with the health warning that I am not an expert on gyroscopes. My reference indicated that the degrees of freedom of a gyroscope include spin. This means that the number of degrees of freedom is one more than the number of gimbals. A subsequent response revealed that although the commonly accepted convention is that degrees of freedom do include spin, the French JAR examiner responsible for this subject had decided that this was incorrect. So in order to pass the examination, students must know not what is generally accepted to be true, but what an individual JAR examiner believes to be true! HUMMMMMMMMMMM???
FINALLY
Because of the way this string has developed, readers might conclude that I have a very low regard for the JAR exams. This is not entirely true. Because a great deal of feedback has been gathered over the past couple of years, the FTOs have a pretty good idea of what is required. Students can therefore expect to be properly prepared for their exams. If some of the figures recently quoted in this forum (91% passes at first attempt ) students can have little to fear.
Most systems, however badly designed, eventually evolve to a reasonably effective state. But the fact that the objectives have no real basis makes them no better nor worse than the old national systems. Improving pass rates do not mean that JAR ATPL holders will have been taught all those important little things necessary to keep themselves and their aircraft in one piece.
[ 20 January 2002: Message edited by: Keith Williams. ]</p>