In circuit or not you are allowed to leave the ATZ.
In accordance with Rule 45
(6) The commander of an aircraft flying within the aerodrome traffic zone of an aerodrome shall:
(c) if the aircraft is fitted with means of communication by radio with the ground, communicate his position and height to the air traffic control unit, the flight information service unit or the air/ground communication service at the aerodrome (as the case may be) on entering the zone and immediately prior to leaving it.
If you leave the ATZ then you have no priority in the circuit!