From EASA web site :
n addition to the member states of the European union, the countries part of the European Free Trade Association (EFTA), i.e. Liechtenstein, Norway, Switzerland and Iceland, have been granted participation under Article 66 of the Basic Regulation and are members of the Management Board without voting rights.[6] There are also numerous working relationships with other authorities.[7]
Looks like if UK leaves EFTA , they will be treated like the others ,e.g Turkey, you can follow the rules, but not part of the discussions .