I'm with JimEli on this - using NVG on crappy night to get the required visual references at the end of an instrument approach can make the difference between getting in and not and means there is no faff reverting to NVG flight in that transition from IFR to VFR.
I always flew with my goggles set up so I could look through them or look under them at the instruments - night coastal searches in crap weather often meant going in an out of scuddy cloud and then back to visual - very disorientating in the dark but being able to gain visual references quickly made a huge difference.
Crab - I agree, but I would also say NVG flying is a blend of 3 visual scans, which vary in proportion to the conditions outside:
1. Looking outside through the goggles
2. Looking outside not through the goggles - (the only way you can discern the colours of obstacle lights, nav lights etc, and the only way you are going to get good peripheral vision cues when T/O & LDG in an environment where you can use white light, or you have a lot of cultural /helipad lighting.
3. Looking inside at the instruments (not through the goggles)
A rule which requires you to take your hands off the controls and lift the goggles when you are reacting to IEIC is a very blunt instrument to deal with the problem of knowing when to transition onto instruments. Its the same problem in the day. The answer is better training in experiencing marginal conditions and decision-making about when to abort. I dont think taking action against someone for leaving their goggles down whilst flying in IMC actually improves safety.