Getting gear and flaps and speedbrakes out beats the drag of speedbrakes plus drag increase from higher speed. Every glider pilot does not have the huge amount of form drag that you can introduce on an airliner.
And this, when you have more drag but a lower limiting speed in a different configuration, is when it does indeed start to require some thinking about. You do however get max angle by flying at the limiting speed in a given configuration. If we decide to get into the issue of which configuration to use, we have to turn this type-specific.
Then there are those other factors as well, as you rightly point out. If you just get draggy and keep your speed, you will have a nice stable ride and avoid those speed changes. Chances are the benefits outweight the potential energy loss benefits of diving... especially in the TMA. It may well be better - but the maximum angle of descent of the thread title it won't be.