VH-XXX:
Albury is in a class of it's own. I recently tracked over Albury in CTA. They asked if I had a VOR. I said "I have VOR but haven't used it before as I have never flown this aircraft before, thus would prefer GPS" They responded that I HAD to choose my outbound VOR radial to obtain a clearance and that I HAD to use the VOR. I stated again that I was fitted with GPS, to which they replied that I wasn't allowed to use it.
and
If that's directed at me, 6,500ft VFR in an IFR aircraft
That's because GPS can only be used by VFR flights as a supplementary aid to visual navigation and not as the sole source of navigation, so AY tower is technically correct.
AIP indicates that when flying VFR, a pilot has the choice of either navigating by visual reference to the surface or through having access to a full time licensed flight navigator, an approved self contained navigation system or a ground based radio navigation system. GPS is not considered to be an approved self-contained navigation system as it relies on externally generated radio signals to determine the position of the aircraft.
So the only options available to you were to accept a geographic tracking clearance (cloud permitting) from the controller or a clearance based on a radio navigation aid (if you are rated to use the aid)
Not realistic or practical in today's navigation environment but, unfortunately, them's the rules!