Andy
I may be misunderstanding your problem. If so I apologise.
At 40000ft your TAS is roughly twice your IAS/EAS. At sea level they are both the same.
Momentum is dependant on TAS. So the tendency for the aircraft to carry on in whatever direction is hugely greater at 40 than at sea level when if at the same IAS/EAS.
Control power and lift however are both IAS/EAS dependant.
So at 40 Mr Newton wants you to carry on in a staight line in spades while Mr Bernoulli has even less to play with to generate a distubing force than at SL thanks to mach effects.
BTW I was nearly thrown off the course at ETPS for writing that the controls were less effective in certain circumstances. It was explained to me that a control was either effective (if it caused something to happen) or was ineffective (if it caused nothing to happen) and that what I should have been talking about was a reduced level of response. Happy days.