I did a reasonable amount of investigating on this topic as I have an Australian PPL and flew G reg aircraft on it up until late last year (when I finally got around to doing a skills test, and therefore now have a shiny JAA PPL as well).
When I first started doing this, the perceived wisdom - with which I had nothing to challenge - was that I could not take a G reg aircraft out of the UK. This very much seems to be the accepted view around flying schools etc. This, however, isn't stated anywhere - nor is it implied.
I did as much digging as I could and still even have the relevant bits of the ANO printed in my headset bag for dealing with people who did not accept what I was saying as the law:
Essentially, ANO (2005)
section 26 para (3) then (4a) states unequivocally that an ICAO license is "... deemed to be a licence rendered valid under this order". There are two things you are not allowed to do - paraphrasing slightly, but (i) is act in a commercial role and remunerated for your flying, and (ii) is flying under IFR in controlled airspace.
Now paragraph (4b) states "a JAA licence shall, unless the CAA gives direction to the contrary, be deemed to be a licence rendered valid under this Order". This is the same wording as stated in (4a) and therefore apart from the two things specifically disallowed, it implies that the "deeming" is equal in both cases.
This is also stated in LASORS, section A7 - but the ANO is of course the authoritative source.
Now I'm not a legal expert by any means, and I'm sure someone here will have a more knowledgeable view, but it does look like the CAA are acting against the letter of the ANO here.
Paul.