What if the weather deteriorates and you have to maintain 400ft for a short period? And you receive a complaint?
Then you might have to explain to the CAA why you continued your flight in IMC conditions without declaring an emergency, or doing something else that's sensible.
After all, if you can't legally maintain VMC conditions, you're IMC.