So long as the cost of the lunch does not exceed half the direct cost of the flight
IMHO it is important to realise there are two quite separate issues here:
1) a CAA prosecution
This is extremely unlikely unless the pilot is running some kind of transport scheme, e.g. taking loads of people down to Le Mans and charging them for it. This has been done many times. Usually an AOC holder sees it and moans to the CAA and they, reportedly, turn up, interview the passengers separately and ask them how much they paid, etc. You can find a list of CAA prosecutions (I suspect they don't list most of the failed ones though) on their website and it's obvious they are extremely rare and 99% of the supposed illegalities discussed on pilot forums are never enforced over here.
2) insurance coverage
This is the one which might actually get you - if there was an accident involving an injured/dead passenger. The reason is that the owner/operator's liability to passengers hangs on the pilot being found
negligent (Civil Aviation Act) and that in turn will be easier if he has been prosecuted for illegal flying. The passengers may be you friends, but are rest assured that 99% of people will forget that if they can get an extra £10k or £100k out of it. I fly an N-reg plane which can't be legally cost shared in UK airspace (ANO Art 140) and while I have been offered cost sharing countless times, I have never accepted it.