If I have deployed the speed brakes, I keep my hand on the lever, and use my ‘outboard’ hand to adjust the FCU if required, or ask the other pilot to change settings for me
Airbus FBW annunciates when the speed brakes are deployed, and auto-stows the speed brakes any time TOGA is selected, (actually if thrust levers are above MCT).