Bit more to it than that ...
Two important limitations relate to maximum airspeed (influences forces generated on the airframe) and maximum mach number (relates to the onset of significant problems associated with compressible gas flow - shock waves, shock separation, mach tuck, etc ...)
The airspeed consideration is limiting at lower levels while mach is limiting at higher levels.
Therefore, it is conventional to fly with respect to the airspeed at lower levels and mach at higher levels.
The practical implementation is that the climb and descent is planned around a particular combination of airspeed and mach number so, on the way up, we start with a constant IAS climb and, when the Mach No. increases to that desired, we continue the climb at constant Mach. Reverse applies on the way down ...