The higher you go the smaller the operating enevlope is. That's the difference between minimum and maximum airspeed.
The following numbers aren't perfect and are just for an example -
Stall might be 180 KIAS, buffet might be 210 KIAS, minimum recommended speed about 230 KIAS, cruise would be around 250 KIAS, and max airspeed around 270 KIAS.
As you go higher you're dealing with decreasing KIAS to fly the same MACH. IE, Mach .80 is 320 KIAS at FL 300(guess), 270 KIAS(guess) at FL350, 250 KIAS at FL390, and 240 KIAS at FL 410.
So if you're cruising at FL 300 you're 90 KIAS above minimum recommended airspeed. At FL 350 you're 40 KIAS above, at FL 410 you're 10 KIAS above.
This is high altitude jet flying 101, especially when you're heavy, above OPT ALT, or near your MAX ALT. It's dealt with tens of thousands of times a day, day in, day out, for decades, by professional pilots.
Book a flight, ask for a beer, sitback, and relax.