If you turn away from the shoreline, you'll instantly be confronted with a featureless, gray void. You'll have nothing by which to judge your altitude or the airplane's attitude.
Except an AI and Altimeter.
If you've received sufficient training to allow you to climb, cruise, turn, and descend on instruments, you might be able to climb above the fog or low clouds to better visibility above. Once you're there, try to establish contact with a flight service station or an air route traffic control center for assistance in establishing your position and setting a course for an area with better visibility.
'Suffiicient training' is not a black art, nor is maintaining sufficient proficiency and is far more sensible than trying to navigate in a 'featureless gray void'. All very well to be taught to turn around BEFORE you get there, but these CFIT accidents just go to show how many get caught out with no options left. A little more I/F training and training when to recognse an emergency when I/F is appropriate, would open another potential option. The PIFR is a still better idea.