If you are at an uncontrolled field with A/G service, and you are doing circuits, it is retarded to set a given QFE if it shows your aircraft being underground or 100ft up whilst still stationary! A transmitted QFE is a 'best effort' for pilots joining the circuit, especially at a lot of GA fields where it is derived from someone setting a ropey old aircraft altimeter (note - calibration for position error will be somewhat out of whack!) in the clubhouse - to ZERO.
As the legal requirement for altimeter accuracy is 2mb this gives the potential for 2 perfectly legal altimeters to be up to 120' out with each other! If everyone sets zero on the ground then everyone's altimeter will read circuit height at circuit height, and anyone joining on a transmitted QFE will be in the right ball park -ish. Anyone joining and then doing subsequent circuits would be best off setting zero on the first touch and go, presto! Everyone flying on EXACTLY the same reference - HEIGHT ABOVE FIELD ELEVATION.
As for the point about sloping airfields, most will employ a circuit altitude based on QNH anyway, for precisely that reason.
Of course if you're flying somewhere a bit more upmarket with ATC and decent equipment it's a different matter, and with a lot of my students I end up covering up the whole panel to stop them chasing the needles instead of flying the aeroplane!!