Although you mention PPL you need to know that there are two licences you can get in the UK: the PPL and the NPPL (National Private Pilot's Licence). The NPPL is a national licence rather than being based on the international standard as the PPL is. Therefore you cannot use it abroad as you can the PPL. The training is a little shorter and a bit cheaper, but don't even consider it if you want to fly outside the UK.
The PPL will allow you to fly a UK registered aircraft anywhere in the world. It will also allow you to fly foreigh registered aircraft in the UK (eg those registered on the American register). A PPL will also generally allow you to go to Europe to those countries which are signatories to the JAA (Joint Aviation Authority) and hire locally, ie an aircraft registered in say France to fly in France.