I frightened the living bejezus out of myself on more than one occasion as a young buck while winter flying west over the mountains from Bankstown/Camden. It's very hard to put older heads on younger shoulders without having the odd fright along the way ....
There will always be some VFR scud running activity. One very simple rule has stood the test of time. This was hammered into my skull during CPL training, especially if you fancy running up a valley. If you can't see the clear on the far side, with whatever depth margin you want, turn around/divert. Right then .... no ifs, buts, maybes.
The daughter of one of my parents' closest family friends lost her son and his mate when they didn't follow that simple rule doing the same sort of flight that I and many others have done in the same area .... The sad thing in that prang was that both of the kids survived, although injured, got out of the wreckage, but died in the following day or two. Flight plans and beacons have a lot to recommend for getting out of strife.