I agree almost entirely with BlueEagle.
When the A/P is engaged, the PF monitors the autosystems, and the PNF gets on with his other tasks - getting the weather, updating the plog, talking to company or handling agents at destination, checking cabin crew are happy, talking to the SLF, etc. etc.
When PF hand flies, PNF has to have almost all his attention on monitoring the PF, and has to be listening to ATC. Other jobs threfore do not get done, possibly being postponed until they are urgent, and PNF is then working his socks off getting everything done, and is under stress. This is not a good idea. Stressed pilots make mistakes.
In a busy, complex TMA such as London, Paris, and many others around the world, hand flying is a luxury that we cannot afford.
Elsewhere, such as a simple procedure into a quieter airfield, there is nothing to stop the PF briefing the PNF on his intentions, asking if he minds the later stages being hand flown, and keeping his handflying skills brushed up.
In that last paragraph there are two very important points. One is to maintain CRM - ask the other guy if he minds and ask him in a manner that allows him to say no. The other is that it is important to keep your handflying skills maintained. Sod's Law dictates that if you let them get rusty, you will get an A/P fault that you can't handle. Part of being professional is maintaining ALL our skills.
Lastly, if a F/O as PNF is getting behind the game, he needs to be encouraged to speak up and ask for the A/P to be re-engaged. So the PF skipper needs also to maintain a monitoring watch on his F/O. This is more difficult when handflying, and needs more effort. Such effort is a necessary part of command.