The main reason is to avoid inadvertent pressing of the TOGA switch affecting rollout during an automatic landing. People do confuse switches as anyone who has seen someone press TOGA when meaning to disconnect the autothrottle on short finals will testify!