Why not ban the catholic church because of the Inquisition?
You are confusing something that happened in the middle ages at a time Normans would slaughter a village of peasants because their sisters cousin forgot to give them a birthday present.
The Nazi swastika is distinct in it's presentation, so has nothing to do with the other variants.