A reasonable rule of thumb would be: Any time that you are anywhere that you could expect to sit down you take your hat off. Doesn't cover every eventuality but covers most.
Can't help thinking we are going to get the anti-hat brigade having their say here quite soon!