With the possible exception of KEnsington palace, these are hardly palatial palaces for the service chiefs. I have visited one for an official function (as a very junior supernumary) and was astounded at how frugal and threadbare it was.
If you look at the diary of a 4*, you will find it is full, usually 7 days per week, and they work very long hours, ranging from official breakfasts through to late night functions, and are also expected to do a busy and challenging job.
The role of the retinue is to provide a driver, a cook, a valet and ensure that their VSO gets to the right place, in the right rig, and is able to host some very senior foreign officials in a reasonable way. The food is not luxurious foie gras, but normal officers mess type cooking.
We may grumble, but having seen first hand the importance many foreign VSOs attach to building good working relationships with their UK numbers, and also the way in which we expect them to represent the UK and their service, I would rather a small amount of public money was spent on this, rather than some dire 'PC' project. The value the UK gets from this silent diplomacy is incalculable, but makes a real difference.