Pretty much as you said really.
There are times on our aircraft when autopilot cannot be used....below a 1000 feet not coupled to an ILS, with flaps at full, below 1000 feet after takeoff etc. As for hand flying, i would say most hand flying takes place in the cargo world as you dont have passengers to worry about when you are throwing it around compared to the auto pilot which is very smooth. Apart from on approach, i wouldnt say there is much hand flying at all in the pax world.