Getting out of the soup
That's a great description of the conditions on the day Jaba; gives a really good overview of how easily one can get caught out.
It's very easy to say "if it's not VFR just don't go", that's a statement that would seem to suggest there are two weather states, one okay for VFR flight and one that requires IFR and IFR only. Of course, as your post indicates, things can start out very well and deteriorate around you on the way. And what appears to be a situation you can deal with can then turn nasty without you realising just how bad it is. The other factor is the pressure Des would have been under (which I've referred to in a previous post). They were already a day late with, I believe, a plan to have come back on the Sunday afternoon, and it no doubt may have seemed like it wasn't going to be too difficult to get around the bad weather as he approached it.
I was once flying VFR coastal around Eastern Victoria (somewhere near Mallacoota) just on 500 feet AGL travelling north along the beach in a Beechcraft Sundowner (not a fast machine, we had a 20 knot headwind...cars on the road below appeared to be going faster than us...) The weather was marginal with low ceiling just above me. The forecast was for improving conditions further to the north so I was believing better weather was not too far ahead. I kept seeing patches of blue above me until I (incorrectly) surmised the cloud was breaking up. I decided to climb believing I'd be able to get up a little higher and be free of the low ceiling that appeared to be breaking up above me. I pushed the throttle forward and climbed thinking I'll be above 7/8 or better in no time. I went into cloud (I'm not IFR rated) and I didn't come out until 3800 feet. In brilliant sunshine and crystal clear air I was then flying above a sloping cloud deck, which could have been a disaster in and of itself but I'd read about such things, knew my A/H was working fine and decided I needed to believe the instruments (but clearly I was in an environment I very much knew I shouldn't have been in).
After breaking into the clear I decided the most sensible thing to do was point the aircraft due east (knowing there was no land or mountains until at least NZ) and descend back down at around 400 ft/min until I popped out over water, which I successfully did about 15 miles offshore. I then did a 180 degree turn and headed back to the coast to continue my trip at 500 feet under the ceiling.
It was a defining moment in my flying experience. Fortunately I'd only just recently done a BFR where I'd done about 30 min under the hood so my instrument scanning skills were recent. It taught me a big lesson about believing cloud above you is breaking up or that it's not thick. What I think I did well though was kept climbing after I went into the cloud rather than trying to come back down again when I was near headlands and hills along the coastal stretch and already so low, maintained good instrument scan & control, pointed the a/c away from high ground using very gradual movements and used a low rate of descent to let down back out of it.
It pays to have a plan on what to do if you do go into cloud suddenly and find yourself not breaking out within a few seconds and I'd encourage all VFR pilots to really think about it.
Ren