Or is there evidence that they interfere with more than just the VHF radios?
It's not just that. When airborne you are in line of sight with too many GSM masts and I've heard that this may cause problems because your GSM is trying to contact too many masts at once. Overlapping frequencies, that sort of thing. Apparently to the point where you might bring a whole area down.
I've never seen that proven though.