Indeed - it's one of the hazards of Priority tagging of bags as well - anyone with malicious intent will naturally target bags with First/Priority tags on on the grounds that they are more likely to have valuable contents.
Having said that, the only time this ever happened to me it was a genuine mistake and with the assistance of the airline I got my bag back within a few hours (and the person who took mine got the almost identical bag that they left on the carousel). Fortunately it was Cathay involved. Lesser airlines would have shown no interest at all.