It's like standing on the top rung of a step ladder.
You can do it, and as long as you're careful, you'll be OK.
But there have been a few people who've not done so well by it, so the manufacturer puts a warning on that says "don't do this".
If the manufacturer puts a warning on it that says "don't do this", and you chose to do it, then you're the one responsible if something goes wrong.
It seems to me that most small plane manuals I've read say "don't retract flaps on final".
Interestingly though, when just a couple feet above the ground, I was taught to "dump" the flaps in order to lose lift and land much shorter in certain cases (tailwheel, very short field landing). But I was being trained for "bush" flying, and I wouldn't recommend it on planes with "normal" landing gear (by that I mean used to normal stresses).