the way I see it is we are essentially asking the question:
Is the ground speed advantage by allowing the wind to push us off course sufficient to offset the extra distance we will travel across the ground?
I dont doubt for a second that we wont have a faster ground speed by not accounting for drift, in the end when we account for drift we are flying with more of a headwind (or less of a tailwind) than what we would if we just flew the heading. If we think about the wind triangle, it takes us the same time to travel the adjacent line as it would the hypotenuse. By choosing to stay on track we fly the adjacent line, if we let ourselves drift then we will fly the hypotenuse but the time spent is the same. Hypotenuse is longer and that reflects the extra groundspeed we will have from experiencing less headwind. The second half of the journey mathmatically is exactly the same as the first, just mirrored the situation.
I think that we will end up at the same place in the same time.