Absolutely right .
Any suitably organised outfit with the required level of expertise and an acceptable exposition could do the PFA's job in terms of responsibilities in respect of virtually all non-microlight permit to fly aircraft.
The real point is that no such organisation currently exists other than the PFA which does hold the necessary approvals from the CAA now.
It has taken over 50 years to establish the credentials of the PFA and respect world wide for the level of technical expertise they have in house and through their network of approved inspectors and test pilots.
Any alternative organisation might find it an extremely difficult task to reach a similar level of fitness to obtain and hold any official approval within any acceptable timescale.
The devil you know is better than the devil you do not especially if it is a democratically controlled operation where members can have a direct influence on policy.
Trapper 69