In my experience dirt's effect is small, but those bug carcasses that create bumps make measurable difference in power and also in stall speed, and sand erosion is even worse.
I agree that soap and water, and a soft non-abrasive scrubber do the trick at the factory.