The government can also introduce into the Parliament and pass stuff called "legislation". Governments don't seem to have too much trouble in producing lots of it.
CASA exists because of one section in one piece of legislation: The Civil Aviation Act.
That Act can be changed. CASA can be given different powers and functions.
That Act can be repealed. CASA would cease to exist if it were repealed.
That is the ultimate and absolute control.
(Soteria: Because Mr Truss and all his predecessors in recent decades don't want to be seen to have any means of control.)