Hi tommoutrie,
I've nothing against your reasoning from a strict performance point of view, except that what has become a habit in day-to-day operations may affect your actions under stress in the unlikely event that you do have an engine failure at V1. From that point of view it may not be a bad idea to maintain flap and thrust setting until at least 400 ft.
I've joined this forum because I was interested in the causes of the AF447 tragedy. It is still mystery why that pilot pulled the airplane into a stall, but one plausible scenario is that under stress he applied a procedure that he was taught in training.