It's a dynamic phenomenon - depends on the rate of change. Gusts, updrafts/downdrafts are short-term and DO affect IAS since inertial speed (i.e. groundspeed) cannot change instantly.
On the other hand the downwind turn is a longer-duration event. The same physical laws apply, but the rate of change is much slower - at least in a relatively fast plane at a standard rate turn. Energy must be added to increase the groundspeed, but it's all within the everyday minor turbulence.
The problem arises in a slower aircraft in a faster-than-standard rate downwind turn. In order to increase its inertial speed, energy must be added at a fairly high rate; either drop the nose or add power.