At 6'6" I don't particularly like it when people recline but they have paid for that button so I can't truely expect them not to use it!
If someone shoves their seat back quickly onto my knees then I can't help it when they can't recline fully but normally I'll ask them if they mind putting their seats forward for 10 seconds so that I can get into a more comfortable position (normally sticking my knees into my wife's leg space or the aisle).
The thing that annoys me the most is people keeping their seats reclined during meal times (as I have paid for my meal and can't eat it when their seats are back as the tray table won't go flat).
This is closely followed by people who get up to have a walk around / use the loo and keep their seats reclined.
Personally I try not to recline my seat; mainly because doing so is uncomfortable on most flights (as my knees go further forward) but also because I know how uncomfortable it is for the person sitting behind!