I don't think Pilotless aircraft for passengers will replace the infrastructure we have now. The current risk is acceptable and can only improve with technology. Eventually, they might be implemented for different requirements such as low-level short distance city 'taxi' travel - but I would suggest that's centuries away.
Where I think we will see them is in the local goods transport area - already being worked on by Google and Amazon.