It was the red poppies which grew over the graves of the fallen soldiers of WWI that inspired the poem, "In Flander's Fields"... in time, the Remembrance Day poppy has become the world's most recognized memorial symbol for soldiers who died in conflict.
Unlike the flags or insignia of individual nations or religions which are exclusive, the poppies simply grew over the resting places of the dead. It didn't matter what side a soldier fought for, or whether he perished with a bang, or a whimper.
If anyone can think of a more appropriate symbol to remember all fallen soldiers, I'd like to hear what it might be. If anyone is offended by the poppy, or is offended by those who choose wear one, now may be a good time to reflect on why.