I suppose it all comes down to perception and mindset. If you already have it in your head, that people of a certain look possibly are terrorists, then any behaviour can look suspicious. Even perfectly normal behaviour.
I would even suggest that those people, who demanded the two men leave the aircraft, are emotionally unstable, and, in the interest of safety, should not be flying anywhere.