Maybe I'm just an old fart.
If you were to offload a colleague for not bringing a hat, you would probably (in this age of instant messaging) find that would be the last time someone wore a hat when they were rostered with you.
Hats make you look like a security guard. So each to their own, old fart.