What really failed was when some prat was allowed out his padded cell to sign us up to this horrendous mess in the first place.
His name was Edward Heath. He later confessed he had to lie to the public (~1973) because they would have never voted Yes if they were told the truth. It was obvious to anybody with intelligence that a "common market" would eventually lead to a central European government, simply because there was a number of countries with poor working conditions (Spain, etc) who would "unfairly" compete with the high labour cost ones like Germany.
Then I can finally have my European licenses lapse
On current EASA proposals you can be N-reg and based in the EU but you will still need to have valid EASA papers, unless you (the operator) live outside the EU (e.g. Jersey, Croatia, etc). So if you fly just VFR, keep your EU license+medical valid.