The fact is that if you wear uniform in the civilian environment, you draw attention to yourself and like it or not, that makes you a potential target.
That said, it doesn't really matter because the vast majority of military personnel, regardless of rank, could wear uniform, pink pajamas or nothing at all and, in the civillian environment, they still become very obvious after a very short period of time.
Sorry chaps, you must try harder, like it or not, those who may be out to harm you know that too.