You could make the same argument about overflying forests, etc.
Biggles was a master of tree landings
wonder what happened to him ?
Flying is all about risk management. I try never to do anything in aviation where I do not have an out. You must have other doors open to you or it all becomes a game of Russian Roulette.
Flying over extensive fog banks in a SEP is a no no in my books as in the unlikely event of an engine failure your options are closed and you are in the lap of the Gods.
A couple have posted 1000 feet agl as a sensible minimum for SEPs as that would give a reasonable chunk of VMC below to take to a field or other landing area.
Anything less and its still possible but not clever if you break out at 500 feet over an extensive built up area?
So it really depends on how much risk you are prepared to take?
I can remember a few years ago in a twin taking off from literally the only clear airfield in the uk with the whole of the UK covered in fog.
Soon after I took off even that airfield went down in fog.
Aberdeen was clear as was the whole of Ireland and I had plenty of fuel to divert to Ireland. Ok I did the trip but would not have contemplated it in a single.
So really its your own risk management and how much risk your prepared to gamble with?
Pace