Beside the issue of the flight deck door, I wouldn't like a whole aisle of passengers to see me sat on a metal canister having a break or even worse...being seen eating something that is far nicer than what we serve to passengers!
Another point, irritable passengers or medical reasons. If we have to deescalate a situation or deal with a medical incident that can be dealt with out the view of passengers, this leads to far better scenario than to have the performance in front of all the passengers!