It is indeed during the extension of speedbrakes on the A320 and can be heard at ground level with the aircraft at 8000', maybe higher. The crossing height at my house was +8000'.
It was as regular as clockwork. At 2330 the A320 from PER would wake me up as it was cleared below 8000' with "expect track shortening" for runway 16 at MEL. I never heard the plane, just the weird whistle sound as the chicken stick was used to rectify the profile.