If you don't pay the VAT in the UK the aircraft is not in "free circulation" and liable to be stopped and checked/searched all over the place. How much travelling do you do?
Keeping it on the 2REG does not actually stop you paying the VAT if :
1. You have the right to register the aircraft - if you are not a CI resident, you may need a local corporation.
2. You intend to operate/keep it inside the EU for any length of time.
Much like plenty of N registered aircraft have paid the VAT and entered into free circulation............
The costs for putting an aircraft on the EASA reg (unless it is permit, of course) are many and varied....how long is that piece of string. It depends on the age, type, level of non-original equipment and how up-to-date the maintenance is. You would also need an export CofA - something I am not familiar with from the CI. Although I don't suppose they are difficult to deal with or nobody would use them.
I wouldn't recommend messing with the French authorities; they tend to turn up mob-handed (big boots and guns) and interrogate anyone on the aircraft, as well as demanding all the documentation. Slightly limiting one's travelling if it's not all in order.