Vincent, apart from your climbing turn bleeding off airspeed, and the natural tendency to want to pull the turn tighter as you get blown downwind, there's also the fact that the wind tends to increase as you get away from the ground. So with your climbing downwind turn, you quite possibly experienced an increasing tailwind, which is a genuine, nonmythical hazard when your margins are already thin.