"But we gave up our territorial ambitions and don't really want them back"
And yet since 1991, we have operated in a long list of countries, many of which were either former territory, or fell into our spheres of influence. Is it any wonder people are suspicious of UK / western motives when they see us still hanging round in areas we'd claimed to have left 40 plus years ago?
We may not harbour territorial amibitions, but we still have policy ambitions to keep certain nations in our sphere of interest / influence.