Guys, this discussion re TOGA or not after engine failure is a perennial one. Already some great points have been made in both directions.
As Vilas said, "consider" is the mantra. But so many times in the sim I see pilots forgetting to "consider" TOGA and so my personal SOP after trimming is:
AP1/2, HDG Pull, TOGA Thrust - I can always consider not selecting TOGA if I have a good reason. But why would you not want to double your rate of climb?
I hope you would all agree that pilots tend to rotate carefully and more slowly after an engine failure to avoid over-rotating and losing speed. So consider a bad day: wet runway, perf limited by a close in obstacle which you are going to clear by 15' (Okay, that's net). Because of the EF you have rotated slowly and so are already behind the expected performance profile. Additionally, you are now flying fast on V2 imp and so are experiencing more drag than expected. All in all, you are in bad shape. TOGA will help here!!
These are only my thoughts and I certainly have no intention of starting a tiddling competition over this but you are welcome to comment.