Greg
I think you answered your own question. It's all relative.
Density of air is directly proportional to its weight.
In a column of cold, dense air, as you climb up into that column of air, you lose density/weight (its now below you) faster than you would if you climb up inside a column of warm, less dense/heavy air.
So if a cold column of air (say) 10,000 feet high, weighs 100kgs, and you climb 1,000 feet you lose 10% of 100kg = 10kgs. If you are in a warm column of air 10,000 feet high - say it now only weighs 80kgs. If you now climb 1,000 feet you still lose 10% = 8kgs So you are not removing weight as fast in warmer air.
Remember that warmer air has lots of more active air molecules, and because they are more energised and bouncing about, there are less of them in a finite column of air at any one time (they've bounced into the adjacent column of air) = less density of molecules = less weight.