At mealtimes I understand and would never put my seat back during the service.
The rest of the time...some of us who are 6ft 2 find that extra bit of recline adds to the comfort level....being upright for a 12 hour flight in economy is a wee bit much. Do I have to ask a person if his answer is going to create 12 hours of discomfort for me?
If someone reclines in front of you and you don't like it..not a lot you can do really. I consider it a right (except meals).