Short memory
Dick:
"I have found when they man the tower for a few RPT flights- say on a weekend when the fighters are not flying - they also activate a number of restricted areas."
It isn't only the fighters in Willytown that use that airspace. It could be anything from UATs, Wedgetail using the airspace prior to departing land somewhere other than Williamtown, to visiting military even carrier based aircraft. So the associated restricted airspace can be active even though the tower is not active.
Your interests might be better served to concentrate on the Williamtown coastal route rather than questioning every bit of military airspace. You don't know because you don't need to.