The scenario you've just described was normally when I'd get told my ATC that they'd seen me vacate, or when they'd ask me to confirm vacated
I agree with the need for situational awareness, it's why I'd always report vacated whether asked to or not. Yes, it may take up a teeny amount of RT time, but at least my conscience is clear knowing that tower had a clear-cut answer as to my intentions. I never had anyone in the tower raise concern about my practice.