"Shooting off through the night as a VFR driver with googles doesn't sound to fantistic an idea...."
It certainly ISN'T a good idea! It's extremely dangerous because when the goggles finally clamp down when they have reached their limits due to to cloud the pilot has possibly no other escape - he's left behind his option for naked eye / visual contact flight. This is a long known risk, which is why the weather limits musn't be lowered for NVG ops at night and the pilot / aircraft combination MUST be able to safely fly IFR if required.
Those of us teaching this banged on long and hard about this to our RAF hierarchy; finally they saw sense and night wx limits were aligned, even though all of our pilots were instrument qualified. It worries me greatly that other organisations might like to think they know better, or are reluctant to learn from the mistakes of others, even though their pilots are possibly less experienced........