1. It can’t and doesn’t produce across-the-board competency, only the achievement of minimum standards
2. A few
3. Generally the latter, but you have to ask the same question of the examiner after the same trip – he often sees it differently.
4. Occasionally, but they usually get it.
5. Generally, but too often they are either just good or senior pilots; they need to be good proficiency checkers, which is a different skill-set
If it’s competence you’re studying – then aside from the individual, it must lie between the NAA, the training organisation, the customer and the contractor.
NAAs are too small and get too little exposure to pick up individuals. And they are really only civil servants ensuring that the law is obeyed – achievement of the minima.
Training organisations can’t become known for failing most check rides – would you spend you money there? Can’t hope to place responsibility with them.
Customers can’t really be held responsible because there is no standard of competence which they can demand beyond hours totals. They can write a contract requiring training and testing processes, and usually rely on legal minima for guidance, the but they can’t keep their own pilots to test for competence. So they are stuck with auditing processes, which ensure sufficient exposure to training and testing to produce average pilots but don’t usually maximise potential.
Operators are the ones who know, but in my experience, their standards depend crucially on the standards and expectations of the accountable manger. If he doesn’t recognise or value competence, it rubs off; he simply won’t put the time and effort into testing for it, and won’t know when it’s falling short. It’s not a profit thing – it’s a belief. To implement it needs TRE/line trainers etc who understand that their role is to manage other pilot’s abilities, who can recognise competence, and who know how to act when they find it lacking. These TRE/line training roles are for pilots with great TRE/line-training skills, not great flying skills, and not necessarily seniority. I like Geoffers’ IAM kite, but I’m not sure those that need it will either bother or dare to risk a voluntary external assessment; it needs an incentive, like 10% off the insurance premium.
If taxis are an analogy, BSM will pass out all the students they can attract, the DoT won’t do more than check that the driver is licensed, and the fleet controller will know who’s good but has little incentive other than to sack drivers who dent his cars. The passenger has to take pot luck, and mitigate his risk by using a reputable company or pay less and take a chance which nine time out of ten pays off. Competence is down to the professionalism of the individual and is helped by a management that sees business growth through greater competence (ie careful recruiting and extra training and support). Perhaps superior competence in the majority of drivers is just a beautiful pipe dream; it certainly appears so!