It also doesn't look like the JAR questions are properly validated prior to use.
There are criteria that must be met before a question can be considered acceptable. These include things such as:
1. Does it test the intended syllabus topic?
2. Level of difficulty. There are a number of standards eg 70% pass rate from the intended candidate population. Does the question achieve the required rate. It could be too difficult or too easy. An example would be an ATPL level question being used for a PPL exam, or a PPL level question used for ATPL etc.
3. Correct grammar/spelling & is understandable.
4. Only one correct response that meets all conditions set in the question stem.
etc etc (can't remember the rest. I last studied it 15 years ago <img src="redface.gif" border="0"> )
If the difficulty level hasn't been tested on a statistically valid sample of the candidate population then the exam must be graded on a curve ie bottom 20% fail or whatever - until such time as enough data has been collected to allow the equivalent cut-off percentage to be used eg 75% minimum to pass the exam.
Question design & validation is just about a field in itself. Would anyone like to take bets about the validation process that the exams have undergone?