To be absolutely and completely clear:
UK to Continent - you can depart from anywhere (your mate's garden), with no more paperwork than a flightplan. You must arrive in an airport of entry/exit in France, Belgium (or further dependent range/destination).
Continent to UK - you must depart from an airport of entry/exit (see above), but can land in your mate's garden in the UK. A flight plan is again needed, but for UK entry you also need to file a GAR - I just email mine, using a pre-filled excel template.
I find the paperwork takes me less than 3 minutes for leaving the UK (a flight plan), and less than 5 for entering (flight plan and GAR).
No stress, extremely easy - and I hope will remain so post Brexit!
Cheers, Sam.