The obvious answer would be the USA. But I've also heard great stories about South Africa and Australia/NZ.
In all cases you'll probably be flying a locally-registered plane, for which you need a locally-issued PPL. Under ICAO there is a requirement that countries offer some sort of validation process which usually means some paperwork. Plus the UK CAA doesn't release your personal details to a foreign CAA except with your permission (privacy regulations) so you've got to give the UK CAA permission plus some dosh to release your details.
I once heard that for South Africa you also needed to sit the local air law exam, but I'm not sure that's still the case. And Australia has had a reputation, for a while, to be very tough security-wise. Again, not sure what todays situation is.
Furthermore, you will need to do a checkout with the rental agency. In case of the FAA, that actually needs to be a full BFR (1 hour ground/1 hour air minimum) instead of just some circuits. But in any case it's a good idea to use that checkout to the fullest to pick the brain of the instructor with regards to local issues.
There was an article in Flyer magazine about this, three or four issues back. Lots of useful information in there.