Flaps on C172 are Fowler, and increase wing area as well as lift and drag. A go-around from a flaps 40 approach demands flaps be reduced to 20, as that last 20 - 40 is nearly all drag, the last 10 a LOT of drag. It also demands a very strong forward push on the yoke to maintain attitude until the drag-flap (40 back to 20) is retracted and the aeroplane re-trimmed. I imagine this is why Cessna limited full flap to 30 on later models - a retrograde step IMO. Just about the best feature (it an't got many!) of the 172 was that 40 degrees of Fowler flap.
Reducing from 30 to 20 will remove mostly drag, not reduce much lift, and not reduce the wing area so is a safe thing to do on final just as long as you are sure the retraction stops at flaps 20 and not beyond!
However it is poor technique and indicates poor approach planning. You shouldn't commit to flaps 30, still less flaps 40, until the point in the approach where such a setting is appropriate.
I only remember once retracting flap on final. It was in a PA38 at Alderney. I knew the threshold was near a cliff edge, and with the fairly strong wind there would be a very strong downdraft on final where the wind flowed over the cliff edge and down. But the 'clutching hand' that pulled the aeroplane seawards was far stronger than I had anticipated. The simple flaps on the Piper were retracted to first stage as full power was applied and we made the runway. I doubt we would have with full flap left extended.