For acceleration, you are correct.
For decelerations, you never get slower than the flap bug for the current flaps (min speed for that flap setting). So you slow to or at the UP/ clean bug, then select F1. Now you can decelerate below UP speed after the flaps have extended to 1. Select and wait for flaps 5 prior to dropping speed below the flaps 1 bug. Etc.