Down here in Oz all sorts of weather conditions could cause problems - from snow and icing in the south (and in the case of icing, not only in the south), to tropical cyclones in the north.
Other problems can include bushfire smoke, which is not uncommon in the tropics during the dry season (winter).
Unexpected loss of visibility can be unnerving. Several years ago I departed the outback town of Coober Pedy on a crystal clear winter morning only to be forced down onto an uncharted strip about an hour later because of low cloud enveloping me from all directions. Ended up pitching the tents and spending the night there - miles from civilisation!
In some parts of Oz conditions can change dramatically quite quickly, so it pays to read TAFs and ARFORs very carefully.
However I reckon the worst possible conditions are not those which cause you to lose visibility, but control of the aircraft. I've already had some unpleasant rides in mountain waves - feeling that I had very limited control over what the aircraft was doing. I also wouldn't fancy being slammed into the ground by a microburst, and therefore give all storms a very wide berth.
Despite all of the above, most parts of Australia enjoy pretty good flying conditions most of the time.