What’s QFE?
When I started gliding in my teens, you set the altimeter to zero on the ground, but as soon as I started flying cross-country, I changed to QNH all the time as the numbers printed on the map actually meant something then, in terms of airspace and terrain, and when talking with other pilots you had a common datum.
I’ve done a fair amount of airline ops using QFE where you had to, made a bit easier by a QFE option in the FMC as well as a metres one. Ditching QFE means one less pressure setting to forget out of QFE, QNH and QNE and there are lots of major airports that are high enough that many altimeters can’t be set to QFE anyway.
If you do all your flying on QNH, you just get used to it. Ultimately, the solution is going to be geometric altitude which will remove pretty much all the disadvantages of pressure-based systems.