In Australia, correct me if i'm wrong) VFR flights don't require a clearance to operate in class E airspace. Therefore, class E isn't really CTA, and since a Special VFR clearance can only be given in CTA, my guess would be no you can't.
besides with cloud at 500ft, you'd be breaking low level flight restrictions