I can't fault the logic qcode, but I also can't find an argument against the bean counters, who also say, "we are paying you for that time, so you stay". Most places in the world now don't have early go's, and so I think if you are getting the odd one, do like Elmer Fudd said and be "Vewwy vewwy qwiet!!".
If you make too much noise they will say that yes you are out of console time, so for that last hour of shift, you amend documents etc. So as i said be thankful that you are some of the very few ATC'ers in the world that still get that perk.