Happened once on a BA flight back from Toronto. Staff did a headcount and realised one too many in Club. Then went for the likeliest person in scruffy dress (i.e me) and asked for the boarding card, which I had by that point mislaid. Once resolved, they discovered its was the besuited gentleman in the seat behind and politely threw him out.