Let me look at the second part of your question, which is when it is ok to press TOGA.
Is it 1, when ATC just starts to pronounce your call sign? This might as well become a request to vacate the runway again or an information about an increased tailwind component, birds on the runway or anything else the controller wants you to know so you can decide whether you are at all inclined to accept a takeoff clearance. Logically, this cannot be right.
Is it 2, when you hear the magic words? May well be; from his side you are authorised to depart his runway. But a clearance for takeoff is not to be confused with an obligation to take off; you can of course reject it. A technical issue might be a reason, as is the question if you have understood his clearance correctly. Now this is not so much an issue in the corner of the world you are used to operating in, but there surely are areas in which ATC is less well understandable, tends to use nonstandard phraseology or even uses a language to communicate with you that it does not use on most other aircraft in the sector. Would you like to confirm that every crew member in the flight deck has understood the same words you believe to have heard? If so, make sure it is so; then this answer cannot be right though. If not, well, go.
Or is it 3., when the PM has read back the clearance? You have heard the clearance, PM has repeated it to the controller, who in turn chose not to call you back to correct what he just heard. So everyone is satisfied that you are good and approved to go.
From a logical and also from a CRM standpoint, it must be thus.