Post the AustLII case law link to where CASA has prosecuted a model aircraft dude, and I will believe you. In the meantime, read about Repacholi as posted in link above. And Bellamy.
PS. Why should a citizen need an exemption to operate a toy - when the toy is not an aircraft in the first place? 
A model aircraft is considered an aircraft, toy or not.
https://www.casa.gov.au/drones/drone...e-safety-rules
There's a link to the rules that were referred to above, you may not need a licence but you can still be prosecuted for failing to follow them. Say your toy drone/model aircraft weighs more than 250 grams and you fly it in your backyard 2km from Moorabbin airport you can be prosecuted. You can read the actual regulations in Part 101, the accreditation dispensation is under 101.374b, which allows you to fly 'toy' drones without accreditation.
SAFA grants certificates not licences. No such thing as a paragliding licence, so it is unlike driving.
It does not matter what its called, if you require it that's the issue if you don't have it. Licence, accreditation or certification just means you have been found legally responsible/accountable to hold that status, and more importantly you are aware of your obligations under law. They are different terms for the same thing essentially. Whether you hold any or none does not change your legal obligations. If I drive a car without a licence and cause an accident I'm held to the same laws as a licensed driver as well as being charged for being unlicensed. The law could change and call it a driver certificate, or driver accreditation it would not make any difference, apart from a bureaucrat spending millions of taxpayers cash changing some wording over several years.