Its nothing to do with o2 leaks that they ban facial hair with O2 masks. Although if you have hydrocarbons on your clothes it can flash combust them.
Its all to do with partial pressure of O2 required to flash burn the rotten food and other deposits in said facial hair.
Suggest you soak some horse hair in bacon grease/dripping and then blow 100% oxygen over it. And see what happens.
Oxygen also has a habit of burning up the flow and can and does set the regulator on fire as well. So you can end up with a burning lump of aluminium and you breathing the fumes in the mask.
I suppose you could say you could have a beard but enforce a degreasing wash of the facial hair before flight.
Didn't one of the Apollo crews get cooked alive in a pure o2 fire while doing preflight checks?