Again, a misconception about the authority of A/G and AFISO services....
Only ATC can give landing clearances. A/G and AFISO can give information only.
Flying Cesspit, you really ought to look up your air law textbook if you don't clearly understand this.
If you are not under ATC, it's your call totally.
CIM
IMHO you did the right thing - delaying your go around further would have been indecisive and making the decision more critical.
If it had been me, I would have been on the telephone to the ATCO asking why the clearance was delayed.
If I was not satisified with the response, I would have wished to talk with his/her supervisor about why causing a late go around is not a wise idea in terms of flight safety.
This is not being needlessy aggressive, ATC would (rightly) expect us to fly their clearances accurately, but they must understand that they have a responsibility to add to flight safety by sensible, early and well communicated decision making; mostly they do in fairness, but aberations should be discussed.