Ozy:
If you're flying manually on raw data then perhaps the MCDU/FMGC can be afforded a lower priority.
Unfortunately not so simple, and that's the problem. In todays aircraft, the autopilot and flight director are all integrated. Both require identical inputs, as they both receive info from the same FCC source (flight control computers). So the PM has to make LNav or Hdg Sel and FLCH or VNav selections as requested by the PF.
It is critical (and SOP) that both of you crosscheck and verbalise these selections before executing, and therein lies the rub. With the PF busy concentrating on his flying and the PM busy doing other tasks (ATC coms, frequency changes, config changes, checklists), it is often easy to miss something, and make the wrong mode selection. I have seen it happen many times.
Today's aircraft do not lend themselves to manual flying as we used to know it. They are not designed to be flown that way as a matter of course. Which is not to say it cannot and should not be done. Of course not. ("Time and place"). But that is why it is recommended to use the automation (i.e PF does his own MCP work) at such busy times to ease the workload and improve x checking & SA (sit awareness) for all concerned.
Ditch the automatics (i.e FD) & fly raw data only? Not really an option in todays RNP airspace. RNAV departures/arrivals demand 99.99% accuracy always. If you are not right on the published lateral and vertical profiles, with associated constraints (if any), you risk violations, with potentially serious consequences. Not worth the risk IMHO.
(p.s the link posted by Maurice Chavez with regard to degraded flying skills does make interesting reading.)