The ol' slippery slope argument eh Keg? Next thing you'll be doing a Barnyardi and mentioning sheep ;-)
Seriously, if polygamists want to mount a case they're quite free to. But that's not the subject of this discussion. It's about monogamous marriage between adult homo sapiens sapiens. Introducing anything further is for the express reason of muddying the waters.
Traditional marriage already has "rights" with respect to who you can marry - with few exceptions we're free to marry the adult of the opposite sex of our choosing, regardless of colour, creed or social standing.