So long as the aircraft is certified for IMC flight, and you have at least a valid IMC/IR(R), no.
You need the right state of mind, too - if you think to yourself "I've decided I'm flying VFR just right now" then it would be illegal, but if you think to yourself "I'm flying IFR just right now" then all is OK.