Just follow your SOP.
My SOP says that it is no problem to use TOGA with flex speeds. It is a bonus.
Others have explained why. But if your SOP says no, then it is no.
(But I think some here might benefit from studying the manual that came with he LPC
)