I always understood that control power was proportional to IAS. At higher alt and temp you will need a higher TAS/Gnd Speed to achieve that IAS but once there you are looking good. As to the loss of thrust, if you have less available, you'll need less control authority.
Might be talking pants though!