You kind of identity the problem and the solution yourself in your first paragraph: "VLS shoots up".
Speed brakes are not a switch. Extend them fractionally, let the aircraft respond, and then slowly and smoothly extend as needed. Once established in the descent you can get close to full deflection (or half with autopilot engaged).
VLS is proportional to load factor so similar care is required during the level off.