My 2p :
Hand flying in a Boeing/Airbus does increase workload for both PF and PM. More talking is required, and PM has to do part of PF's job as well as their own, since PF is not supposed to make their own selections. PM also needs to monitor PF much more closely when PF is hand-flying, which takes up a lot of PM's capacity.
I really can't see the problem of PF making their own simple FCU changes - if PM is busy. Airbus FBW stays where you put it, and a Boeing won't go far off if you have correctly trimmed it, so you can select a new heading or whatever with your inboard hand, while hand-flying. I don't recognise this threat of veering off our heading or altitude if we look away for a second or two. In our cars at 70mph; we don't veer out of our lane or crash into the car next to us on the motorway when we look down at our speedometer or clock or radio - why would we do so in a big stable jet.
If PF has to ask PM for every selection it can get silly. When in busy TMA airspace*, PM is making many radio calls and frequency changes, and reading those back and dialling them in. If PF has to ask for an FCU change, they will either talk over ATC, making PM forget the six digit frequency, or have to wait until PM has finished their task.
*obviously, not a sensible place to hand-fly !
I don't see the problem with flying 'through' the FD in the short term. The FD tells you where you need to be to follow the current settings on the FCU. However, if those settings are no longer valid, and PM is busy with an important task; then I will fly a different heading and ignore the heading bar of the FD while still following the pitch bar. Or vice versa. When PM is back, I can ask for the FCU changes to update the FD. Or turn off the FD. Or - if it is getting too busy, pop the AP back in to reduce workload for both of us.
Hand flying should be encouraged by companies, it makes a lot of sense for them to keep our skills and scans sharp. But they should also specify the conditions and situations where hand flying would be appropriate, so as not to introduce extra problems or reduce safety or compromise the commercial parameters of the flight. Hand-flying a complicated SID - with strict noise abatement turns that gets peeled off into quick-fire ATC vectors and speed changes in the London TMA - is asking for cock-ups and embarrassment ! Hand-flying an intercept to an ILS or a visual approach for practice on a quiet, CAVOK day with light winds when both pilots are alert and rested, would be fine, and should be encouraged.
Companies could have us log 3 manually flown approaches every 6 months, like we used to do for practice auto-lands.