I don't understand why PM's get overloaded when the automatics are off. You have one system less that you have to monitor
If you feel that you need to monitor the flight parameters more than you need to with automatics engaged, then you have too much confidence in the automatics.