The slower a wing flies the higher the Angle of Attack (or "alpha") becomes, which increases drag and eventually results in the wing-stalling.
To allow reasonably low-speed performance a variety of high-lift devices are used which increase lift, thus reducing the alpha for a given airspeed allowing you to fly slower without excessive drag or stalling, reducing stall, landing and takeoff speeds (and reducing takeoff run).
As your airspeed increases, the alpha lowers and flaps are no longer needed to keep the wing from stalling. Flaps produce drag (which at low speeds is offset to some degree by reducing the wings overall alpha), and as airspeeds increase this becomes quite significant (also, most flaps used on commercial jets have a structural limit of around 250 knots) and it becomes practical to retract them.
Excessive drag affects climb-performance by affecting acceleration and speed: The faster you accelerate, the faster you get up to a reasonable speed; the faster you fly for a given angle of pitch, the more rapidly you climb.