how does an experienced pilot paint himself into such a corner...
Very easily....if your flight plan and planned contingencies prove to be inadequate for the situation in which you find yourself then you are well on the path to corner country - and it is very easy to be sucked into a situation that falls outside your contingency plans.
Most of us have pushed on into marginal conditions in the unsubstantiated belief that conditions will improve at some point in our careers...sometimes you get away with it, sometimes you scare yourself and, unfortunately sometimes you get it catastrophically wrong.
Unplanned low level departure from VMC is not a good place to find yourself, no matter how much experience you have in your flight bag.