There is a simple and fair solution:
When flying in controlled airspace (IFR), pilots should have to speak English to minimum ICAO level 4.
When flying in uncontrolled airspace pilots should have to speak to at least ICAO level 4 of the language of the country whose airspace they are flying in. This would avoid the ludicrous situation of non-English speaking PPLs, ultralight pilots, glider pilots etc being forced to speak English in their own country just so that visiting PPLs from other countries can understand them.