Transponder not required in Belgium as long as you stay low.
From the Belgium AIP, Section ENR 1.6:
2.1.2 CARRIAGE
The carriage and operation of mode S transponders with Basic Functionality is mandatory in the Brussels FIR/UIR airspace
for all aircraft operating:
• in class C airspace
• in all MIL controlled airspace
• in class G airspace for VFR at night.
An exemption to this rule may be granted, provided that the request is made before the flight to the authority having jurisdiction over the airspace concerned.
Note that class C airspace in Belgium essentially covers the whole country, in some places from 1500' MSL upwards. Without a transponder you're going to have to duck beneath that.
To land in the UK without going through a port of entry, you need to fill in and fax the General Aviation Report (GAR) form at least four hours in advance to the appropriate authorities.
I cannot find anywhere current&official whether EDFV is an airport of entry (either H24 or PPR). My old Bottlang suggests that it has customs with PN but that info might be outdated. But this page suggests that it has no customs whatsoever:
AIRPORT Flughafen Flugplatz Pilot
Also no mention of customs on its webpage.
But you cannot leave the Schengen area (or the EU for that matter, but that's not relevant in this case) without going through such an airport, or by reaching an agreement with immigration somehow.