I don't fear automation; as you say there are many things that it is potentially better at than humans. There are also things it is not so good at.
But to my mind the principal argument to say that it is not yet ready is reliability. How many times a day do crews around the world cope with failures? Not necessarily anything major, but enough that if they didn't intervene the aircraft would be in trouble.
Only when that figure is damn close to zero will we be ready for a fully autonomous aircraft.