Pilots don't want pilotless aircraft for sure, but I would not put my bet on the passengers.
Time has not come yet, that is true, but technology will move on, and once it reaches a point where airliner operations without pilots will be feasible (e. g. with remote control from the ground as a backup, where one guy suffices to control multiple aircraft in case it would be required), the airlines will drool over the possible cost cuts, and a giant marketing machinery will be started, promising low fares at an unprecedented level.
And passengers will fall for it.