I think you are assuming a continuous slope in the aerofoil-polar, which isn't usually true, there are many aircraft where past a point, the increase in Cl.max with increasing flap deflection becomes negligible, yet the increase in Cd0 becomes quite large.
Thus, whilst L : D is improved to an intermediate flap setting, it is worsened again as further flap is selected - giving a poorer take-off run and climb gradient in most cases. Not a bad thing, we all rather rely upon large flap settings for landing.
This is the reason that many aircraft (including the afforementioned C172 I think, and certainly a PA28) take-off with an intermediate flap setting, and on a go-around the immediate action (following full power selction) is to raise the flaps to that intermediate setting.
G
N.B. I've just discovered that if you type "L","colon","D" you get a very pretty smiley face, and not shorthand for lift-to-drag ratio as intended.