Conflict - Do you not think there is a reason that military flying organisations around the world wear fireproof clothing? Regardless of their stream - be it fast jet, rotary, multi or even sat in the training system on L/A - the only clothing you will fly in is a fireproof suit, leather gloves, boots and (temperature dependant) a fireproof flying jacket. It's not to look akin to Maverick and Goose, or even so you can strut to and from an aeroplane looking cool - it's a simple factor of safety.
Admitted if you're at FL080 and you have an electrical fire you may have a slim chance of reaching the ground before it overcomes you and your clothing, but a simple pair of leather gloves means you can reach the circuit breakers and battery switches to shut them off and hopefully extinguish the fire - rather than losing the skin on your hands from the heat. Not forgetting that because you can't even attempt to control the fire, as it gets hotter and the flames get larger your polyester clothing is going to start to melt. On top of that, as the plastic units start to melt you're going to have fumes in the cockpit - but because you're already burnt there isn't a lot you can do about it except take a good lung full of the stuff, and finally the burns on your hand stopping you from taking control of the yoke/stick, so you are almost guaranteed to have a moment of rapid deceleration shortly after letting go.
Dramatic - probably, but a definite possibility. Whilst nobody can tell you what to, or what not to wear - it seems daft to not shell out a relatively minor sum to get yourself some kit that potentially could save your life. Agreed safety clothing doesn't give you a magic shield against any and all dangers in the air - but it gives you a fighting chance to do something about an in flight emergency.