If an inspector from any Govt organisation broke into my private building on my private property, then I would have him rightfully charged with breaking and entering.
The fact that Police have to obtain a search warrant to search private property shows that to enter and search private property in an unauthorised manner is trespassing - and damage to private property is criminal damage, no matter who you are.
However, public or designated areas are different to private property, and different laws apply. "Designated areas" are typically aviation centres for pilots and aircraft owners, and workplaces for industry.
CASA inspectors are no different to OH&S inspectors. They are empowered by legislation to conduct inspections, in areas where they consider the laws they are employed to uphold are perhaps being broken.
However, in all situations, "reasonable" is the word used, and "reasonable" behaviour is expected.
Breaking into premises without authority (i.e. - no warrant) and conducting inspections outside reasonable hours does not fall under the category of "reasonable" behaviour.
Damage to another persons property, even under a claim of "investigation", is completely unacceptable, and most certainly a breach of law.