I wouldn't expect 'sh!t to hit the fan' from that; you were told to expect late landing clearance, you didn't get it by 50 feet so you made the decision to go around. You would be in the 'sh!t' if you wheels downed without said clearance, obviously, so based on what you have said it seems a fair decision to hit TOGA.
I'm not sure why the ATCs would have been annoyed in the resulting go round, issuing a clearance after the aircraft has changed attitude is a little late...
Perhaps you could have asked (again) before hitting TOGA, timing is everything of course. i.e. a little earlier than the 50ft decision... Immediately after the "er" transmission for example...
Either way as an ATC I would expect a go round at anytime (even after wheels down), and certainly wouldn't be concerned that you did it, especially if I hadn't given you a landing clearance, mea culpa... Perhaps they were trying to avoid some hard please explain questions themselves... IMHO, there really is no such thing as "no blame culture" in ANSPs, despite the rhetoric of there existence.