As a Brit, you may be surprised to learn that in the rest of Europe there is (mostly, to my knowledge, but perhaps not always and everywhere) a 1:1 relation between one's coordinates in space and the service to talk to.
That is to say, for every bit of airspace there is one and exactly one service in charge. And they'll hand you over to the next one as you fly.
If in controlled airspace, you talk to the owner (CTR => Tower, TMA => DEP or ARR, upper airspace => CONTROL), all of these are indicated on the maps.
In uncontrolled airspace, you can talk to the relevant FIS - for France that will be Lille, I suppose, for the Belgian Airspace it is Brussels info at 126,900 MHz.
For Belgian airspace, everything above 4500' is controlled, which is why I recommend flying there: you will get good service. You may count on being told what frequency to tune into next, often well before crossing the formal airspace boundary.
Regarding Koksijde airspace: I am too lazy to look it up now, but ISTR their CTR becomes class G when inactive, wheras their TMA is indeed taken over by Ostend approach when not active. Check the AIP to make sure, it is freely accessible at wwww.belgocontrol.be