You make a valid point and ATC should release you from the speed restiction in good time ( but sometimes we forget we gave you a restiction in the first place or we get distracted! )
Please say when you need to reduce speed regardless of how busy the RT is , we won't mind. Keeping us in the picture is what matters.