Can I just exchange it for a British one or do I need to do new proficiency tests?
YES, and NO so long as its valid. Complete the following form
http://www.caa.co.uk/docs/33/SRG1136FF.pdf
Could I fly in England (on G-reg aircraft) on my Swedish JAA without having to bother with getting a UK one?
YES, and any other ICAO PPL as well, and not just in the UK, Worldwide.