(1) Australian aircraft operating in international airspace, if there is any consistency between Australian rules and ICAO, which takes precedence?
The Australian rules.
But it’s a false dichotomy.
International Conventions, like the Chicago Convention (happy birthday) bind countries, not people.
Laws bind people.
The laws that bind the operation of an aircraft in international airspace are those of the country of the aircraft’s nationality. (That’s why you cannot lawfully tootle around in a VH-registered aircraft in an active Romeo declared by Australia, 13nms on the seaward side of the territorial sea baseline….)
(2) A aircraft operating within another state, whether or not it is an ICAO signatory state, where there is an inconsistency in the rules, which takes precedence?
Neither. You comply with both, or you’re in trouble.
However, the examples you provided are of differences, not inconsistencies. For rules to be inconsistent, it has to be impossible to comply with both simultaneously. Australian law can provide that when an Australian aircraft is in Roman airspace, do what the Romans do…