Actually EASA have already dealt with a few legacy issue that the UK CAA failed to do anything about for absolutely years.
Ormond Beach which turned into flyeasa. There were literally hundreds if not thousands of complaints about the place and even evidence that skills test were done in an hour etc etc. The UK CAA did nothing.
EASA took over oversight and they were shut down pretty quick.
The only way the CAA will have you is if you play by the rules. If you are of the mind to step away from them they really don't seem to do anything about it.