Nothing to do with the class system, I've walked into pubs in very 'ordinary' locations where the regulars have looked at me as though I had just stepped off Mars and been in upmarket pubs where the welcome has been quite the opposite.
I think it's more to do with the selfish society in which we live, where so many [not all, thankfully] think only of themselves?
H49