Maybe he's confusing it with the sound of a hydraulic pump or some other device. I have heard a Ram Air Inlet actuator on one aircraft type making a buzzing sound (can't remember the circumstances though).
Of course, it's easy enough to check on your next walkaround. Is there any airflow from the ram air exit vanes with no bleed air (the answer should be no).
Having said that, to me it makes sense to switch something off when you're not using it (unless something is energised by turning off the device, like an "OFF" solenoid that might get burned out). You don't normally turn on all the electrical devices in your house at the same time by turning on the main switch at the fuse box. Similarly, on, say, a 747-400, I would turn on the APU bleed after the APU had warmed up and then turn on the 3 packs individually and monitor the duct pressures whilst doing it.
However, if your SOP's say otherwise...