But we're already in a world where pilot error, CFIT, and suicidal pilots result in far more aircraft accidents than mechanical failure.
As the pilot is the last line of defence the failure of the last line on occasion can and does lead to an accident. That line of the defence (the pilot) in the other hand does catch many hardware and software failures and thus prevents many accidents. Accidents that are prevented are not reported. The order of magnitude must be in the order of hundreds of thousands of prevented accidents to caused accidents in favour of the pilot.
It is clear that while operating in the middle of the envelope computers can do a better job than humans. The issue arises when external conditions (WX), procedural or mechanical failures place you at the edge (or beyond) of the envelope. In the case of cars the software defaults to brakes on and STOP. In the case of aircraft it defaults to automation disconnect - manual handling. The difference is significant, the autonomous aircraft will have to be designed with a much greater envelope capability than current aircraft and will require greater system redundancy and a larger number of sensor and sensor types with the added cost that will bring.
The scale of the problem can be seen by looking at FMC irregularities published by both Airbus and Boeing. These are known software glitches that have undesired consequences on the behaviour of the aircraft. In very mature products such as the A320 family or the B737 these irregularities are many pages long. How long then to develop a way more complex system that has no such irregularities at all.
Anyone that flies an Airbus knows that resetting a computer is a daily occurrence to restore normal function. That being the current state of affairs I cannot see a system with the level of reliability required any time soon.
Time is long so no doubt in the end it will happen but its certainly not around the corner.