There is a project being run by a certain chap at a very well known and aerospace-friendly UK university looking at passenger acceptance of pilotless aircraft. The figures indicate that (a) passengers are less concerned about this than one might think, and, (b) the acceptance rate increases year on year. It's a proper academic study with good criteria.
Boeing's 737 testbed flew for many years with a flight deck in the cabin, from which takeoffs, flights, and landings were conducted. The forward flight deck was always manned, and had override switches. History doesn't tell us (or me, at any rate) how often they were used!