We too commemorate the loss of all victims of war. We just do it without the poppies. Like it or not, the poppy is broadly seen as a symbol of British militarism in various ex-colonies, and I reject it for that reason.
Being based in Hong Kong (China) and employing staff from so many countries around the world, I don't see any reason for CX to allow the wearing of poppies anymore than we should allow any other symbolism that might offend.
That said, you clearly have every right to wear your poppy. You just don't have the right to force it down my throat or wave it in my face at work.