gyro,
while I am sure you are fine to fly to the CI, asking the CAA is only half the story on these matters. Because it is an N reg, the FAA is the other half of the story and although the CIs can be considered British for all the reasons given in this post (and so you are ok), you could equally ask the same question of the French. They of course would probably say "that is fine" but in reality the FAA would say "that is not fine". If the FAA said it wasn't ok (which they do incidentally) then insurance would be null and void and N reg pilots in Europe could be flying illegally.
I just thought I'd point this out in case any other N reg pilots are reading this and assume it would be ok to fly abroard in an N reg on a JAA license issued in the UK.