Indeed, the reality of IFR could be perfect VMC. The plane can't possibly know what rules the pilot is operating under, so this cannot be an actual safety issue. It's just the il/legality of it.
What might be relevant to the VFR-only operating restriction would be stuff like
inadequate avionics fo IFR nav
no pitot heat
inadequate gaps to allow control movement under icing conditions
insufficient conductive paths for lightning (esp. applying to composite planes)
etc
But all of these apply to IMC rather than IFR.