For the sake of flight safety we all need to speak the same language and thats not English, it's STANDARD PHRASEOLOGY in English.
You are correct - but that is not the entire story.
The ICAO standard radio phraseology requirement is not the same as the ICAO English proficiency requirement.
The ICAO English proficiency is a measure of a pilot's grasp of everyday English - so it is perfectly possible for somebody with no concept of standard radio-phraseology to be ICAO level 6 (such a person could be any native English non pilot), and it is equally possible for a pilot to be extremely competent in standard radio phraseology yet struggle to reach ICAO level 4 standard.
Testing must be done relative to the aviation environment. I had a Asia student who had studied at Oxford and had near native Englsh skills. She was shocked when I told her she was level 3. Her radiotelephony was terrible and her response to a role played emergency far from adequate.
Some interviewer raters around are academic linguists and have no idea about flight procedures
You are completely wrong. Her radiotelephony may have been terrible and so she may not have met the requirements for an r/t licence, but by your own description, her everyday English skill would appear to be level 5 or 6. I suggest you read the ICAO guidelines:
ICAO | FLS | FAQs
But I agree with you that many native English speakers are at fault for the communications problems that we all encounter because they do not use standard radio phraseology. The worst offenders are East Coast USA controllers - I am a native English speaker and I always find it unpleasant dealing with their impatient, aggressive garbled instructions.