People are expected to start answering a question without a pause, without time to consider an answer. 'So' just a placeholder for 'hang on, I'm thinking of a way to not answer the question I was asked, and if I waffle long enough everybody might have forgotten what it was'. It has replaced 'um' which was interpreted as meaning 'That wasn't covered in my brief, so I'm going to make up something plausible'.