I can just see how a guys professionalism would make him vulnerable in such a situation.
Saving been forced into a diversion in low cloud and vis you'd be doing all you could to stay as high as poss to avoid both obstructions and excessive noise. It is almost a reflex response to fly as high as you can in such a situation. And pinching it at cloud base is just where you lose forward vis while keeping ground contact, so if there's a crane jib lurking in the cloud that's just where you don't want to be. I wouldn't be surprised that if he'd been a couple of hundred feet lower he'd have had sufficient vis to see and avoid. We just don't think of obstructions growing out of cloud when you can't see an obvious base.
IMHO this talk of enhanced lighting is a red herring and would achieve little or nothing if this event was rerun. Lighting is fine in VMC but pointless if shrouded in cloud.
Sadly it looks like an unfortunate operational accident and rushing into legislatative changes is almost certainly going to be a waste of time.
Questions will doubtless be asked re the wisdom of using battersea as a diversion in such conditions given the atrocious conditions at LCY, and even that of setting off from to elstree in the first place, but that's AAIB territory.
We all know the picture when grubbing along in low cloud/vis, and with obstructions close alongside the route extending up into the clag one is in a vulnerable position.
Very sad.