It's safe enough, but you will lose some lift due to the roughness. How much is hard to say, but having the leading edges smoothed seems to help a little. Flying through rain really roughens them. Water isn't that hard, but when the drops hit at near the speed of sound, they can do some damage, and will quickly roughen the leading edge. The metal is thick enough that the roughness isn't dangerous, but having someone who knows what he's doing take a file to it may be worthwhile.