First, I should note that I do read myself, but only if I feel there's time and opportunity for it, and that a lot of flights I do are quite long. On a short flight across the North Sea to London, having only a few minutes in cruise and little time to even prepare for the arrival, obviously I'm not going to be spending any time reading. A flight all the way down to some place in the Canary Islands is a different story however. And I agree with the idea that reading something does a better job at keeping you awake than just staring at the instruments.
To say it's rude, well, you do run out of things to talk about with people at times. And even though I generally get along with people just fine, it's not said I've always got much in common with the other guy, often not enough to have eight hours worth of conversation when flying to Tenerife and back anyway. I certainly don't mind it if the other guy gets a book or magazine out.
A while ago I was flying with a management pilot who shared this view that pilots shouldn't really be reading during the flight. However, he has no problem himself looking through the manuals, or even get out his laptop to catch up with some digital paperwork. I find it quite odd that he refuses me to read my book while he's busy working on his laptop. You're not telling me that laptop isn't a distraction.