Correct, you only need to inform Special Branch of flights to Ireland (North and South), the Channel Islands and the Isle of Man, not othe destinations.
In fact I accidentally faxed a GA report to Special Branch for a France trip (I was given the wrong phone number), they called me straight back saying that I needn't have bothered them!
I have found that Customs and Immigration are very friendly towards to light aircraft making overseas trips.