That was my point.
I've seen guys go flying without clearing the hoar frost and get away with it.
The chances are, you could possibly get away with a small amount of ice on an aircraft not cleared for it.
However, you are entering completely unknown terriotory by doing so, and so the safest bet by far is to make sure you clear all the ice from the whole airframe before you fly, and do not enter icing conditions while enroute unless you have the kit to deal with it.
Plus, you're adding to the chance that something will go wrong, and if it does you're in serious trouble. Even if the accident is not fatal, how is the aircraft owner/insurance company going to view you knowingly going flying with hoar frost still on the wings. It simply isn't worth the risk.