Shunter,
You are indeed correct, the holder of an ICAO licence may fly within the airspace of another ICAO State in an aircraft registered in that State, but not outside the State of which registers the aircraft.
My club did, and still will rent aircraft to holders of (for example FAA licences), provided they do not fly outside the UK when in a G reg aircraft - that is perfectly legal.
That is my understanding of the legislation - but I stand to be corrected if someone is kind enough to point me at the specific legislation.
sooty