Centaurus
You seem to forget that in a lot of the cases of "manual" flight the PF requires the FD's to be giving the required information. That is where the increased workload on the PM comes from, as well as performing his "normal" duties and MONITORING. If the PF is not using the FD's then yes there will be no corresponding large increase in workload. Some companies may also require the use of FD's for certain operations even when hand flying.