Definitely agree Keith. Appropriate objectives are very necessary. Well constructed objectives in turn lead to appropriate questions. Mind you, that doesn't necessarily imply 'good' questions.
Doesn't that open a new can of worms! What objective safety analysis was used to derive data to determine whether or not a particular knowledge area is necessary? None, I should think.
I suspect a combination of each country unwilling to give up some of their most cherished limitations + some grouping of various industry types giving their opinion on what they each believe should be included.
Dick, a slightly absurd example that illustrates what I've been saying would be if those of us who teach aviation theory ie presumably experts in the field <img src="wink.gif" border="0"> , sat a new PPL exam and
our score used to decide if the exam was too 'easy' (or not <img src="redface.gif" border="0"> )
That's where testing against the target population is required. This testing normalises the questions/exams to a common reference eg difficulty level , understandability etc etc using standard deviations.
As for having to go back to the CAA
after the fact to
ask that questions be altered indicates an abysmally poor validation process.
Effectively, the first students to sit the exam(s) were the test subjects in the 'validation' process. They
should have had their exam scores determined using Standard Deviation marking, not the fixed percentage method. After the data was accummulated from a valid candidate sample then the marking can legitimately be changed to fixed score.