Yes - err on the side of caution and abort early. It's not magic, it's common sense but there are those who think they are beyond the rules and believe they are good enough pilots to push on right up to the point where they realise they aren't.
Some just won't listen and until you find a way of making them learn without killing themselves (and others) or stopping them getting into a cockpit in the first place, then we will continue to have CFIT/UCFIT/IIMC fatalities.
As hihover says, many use COCISOS (clear of cloud and in sight of the surface) as a shield of righteousness and protection and blunder on with very marginal visual cues. Special VFR is another misnomer as it implies good weather but in reality is often a COCISOS clearance in all but name.