I wear nomex for my job, but not for pleasure flying. An old friend crashed and burned in a Kitfox a few years ago. He was wearing nomex plus appropriate underwear, his passenger wasn't. Guess who came off the best???? He never flies without a suit now.
It might be a small possibility, but it COULD be you. Why not don/doff at the aircraft to avoid ridicule?