I can't remember the reference but there is somewhere in Part 61 that allows this, it was one of the few sensible things it seemed to introduce! As long as the Chief Pilot holds the correct piece of paper from CASA they're able to check their Pilots on the Aircraft they operate. It's not an Exemption as such, I just can't think of the correct term for it at this particular moment nor the actual reference.
Agree, the CP can check the pilot but not train them to be competent - that's the gotcha!
It's a bit like the flight review - an instructor (grade 1 or 2) can conduct a flight review independently of a 141/142 cetificate holder provided they don't do any training! Now, if the pilot asks you to complete a flight review and their steep turns are safe, but untidy do you demonstrate one and provide guidance, then let them practice til they are tidy? No, this is considered instruction so you'll need a Part 141 certificate with all of the administrative stuff (POH, check lists etc for that aircraft type). Alternatively you tell them to go away and practice steep turns until they are tidy, then I'll look at them again or say they're untidy but safe so I'll just sign you off?
The philosophy seems to be you are considered suitable to judge what's safe (checking), but not how to train them to be safe unless you've got the weight of the subject aircraft in paper to support the training.