Here's some outragous costs the CAA which definately does come within it's remit:
£718 for initial attempts at ATPL theory exams. This is nothing less than an abuse of it's position.
~£400 for initial class one medical - perhaps not as bad, but still pushing it.
£271 for issuing a JAR PPL(A), with IMC and Night Ratings - compare how much the FAA charges for similar services.