It is the engine thrust changing.
Think about it, it sounds like when an aircraft normally flies overhead, but sped up and with a 'whoosh' to it. If it's what I'm thinking of, the first second or so is VERY abrupt and breaks the silence nicely!
It is the changing engine thrust compounding the doppler effect which makes that cool sound and requires a specific change of power setting, altitude from the observer, and relative wind for it to be heard.
People here who are doubting this are probably thinking of the wrong sound. Or I am...