From what I remember, a TDA and a TRFR only have to be avoided my military ac, although good airmanship and a sense of propriety should see civil ac stay out too. An ERF or what used to be called a TRA (not sure about now) would impose restrictions on civil aircraft movements as well.
The airspace controlling authority is the only person who has the right to allow access to any of the air space types listed above and generally an ERF will be controlled by a copper and authorised through AUS acting for SoS.
In the case of ERFs then the copper will delegate airspace control to an ATC authority like with 7/7, 21/7 and various other 'events' that have happened since.
Having worked there, I wouldn't call D&D to ascertain who is the controlling authority because the answer will either be 'don't know', 'ARCC' or they will not be able to give you an answer for Op Sec reasons.