The first 'pilotless' passenger aircraft isn't going to be a 400 passenger commercial airliner - it'll be a private 'bizz jet' general aviation aircraft (or even a prop), certified under far more relaxed regulations than the Part 25 FAA and EASA regs. The safety record for general aviation currently isn't all that great - orders of magnitude worse than the Part 25 counterparts. So you've got some businessman who can justify and afford his own aircraft, but doesn't want to either pay for a pilot or become one. You honestly think the idea of a pilotless aircraft - as safe or safer than the admittedly mediocre level of piloted general aviation - wouldn't be attractive? Sure, some will crash, people will die - just like what happens with all to much regularity in general aviation today. But the bugs will be worked out, and before long unpiloted GA aircraft will be way safer than their piloted counterparts.
Wrong. If there's ever going to be pilotless passenger aircraft (and I'd argue it's at least a century away) the support network and infrastructure behind it will be so large it would have to be first developed by major airlines. A businessman isn't going to just buy an unmanned flying machine, jump in it and safely operate it himself.