Consider this -
For a given speed profile such as Maximum Range Cruise (MRC), and for a given weight (mass), then for every level flown, there is an optimum speed. This will be a constant Equivalent Airspeed (EAS) up to the level where Mcrit becomes a factor (which it inevitably will). After Mcrit becomes a factor, the optimum speed will be a Mach number. This takes care of the optimum speed for each level, the next consideration is to find the Optimum Level. This Optimum Level will be that at which the engine/s are operating at an engine speed for Optimum Thrust Specific Fuel Consumption (TSFC).
Optimum Thrust Specific Fuel Consumption (TSFC) is typically in the vicinity of 90% N, at higher and lower engine speeds the engine is 'off optimum' and fuel expended per unit of thrust produced is greater. Typically, TSFC at engine speeds above optimum N is worse than for the same N variation below optimum speed.
Now, at low altitudes where the air is dense, to fly at the optimum aerodynamic speed, much lower engine speed is required (because of greater air density), thus, although we are doing the best possible
FOR THAT ALTITUDE, we are doing less than the best possible.
The best possible occurs as the air becomes less dense (increasing altitude), and the engine speed must be increased to the optimum (about 90% N) to achieve the required thrust. This level, then, is the Optimum Level, any higher or lower will incur a greater fuel penalty.
If we fly higher into less dense air, the engine speed must be increased to above optimum TSFC speed to provide the requisite thrust. For each % N increase above optimum, the penalty is approximately double that for the same % N decrease if flying lower. Thus, if the Optimum Level is not available, it is better to take 1 level lower than 1 level higher.
There are other factors such as, colder air (which is associated with increasing level below the tropopause) requires less work to compress, thus enabling more efficient engine operation. A good bonus, but the effect is much less than the fact that the thinner air requires increasing engine speed than at a lower level, up to the point where the engine is at optimum TSFC speed to attain the required speed.
Somebody said -
"Because you'll probably exceed Mach Crit." - I hope that you
DO exceed Mcrit at higher levels, that's where Normal operation for Jet aircraft occur (Especially Maximum Range Cruise).