I can't see that the UK has any interest in a flight/s in an 'N' reg aircraft, properly maintained, not based in the UK and flown by a valid pilot approved by the FAA. The FAA are the authority directly responsible in your case. You may require to fly to and from Guernsey and the UK daily throughout the year, for pleasure or business, and why not. There can be no case to restrict you to 28 days a year for private flights.
See: Information Notice Number: IN–2017/029 for a full explanation. You will see the rule pertains to flying UK/EASA aircraft.
VAT in Europe is a major issue for N reg but I'm sure you have already dealt with that.
Last edited by Fl1ingfrog; 13th Feb 2020 at 17:38.