I expect it is still feeling the consequences from Flybe's downsized base over the winter compared to previous years. Demand in the South East is not an issue at all, as you can easily see from the increases reported by the London airports where the catchment overlaps. SOU has been at the disadvantage where around 90% of its passengers come courtesy of one airline. While that airline offers a good selection of routes, the airport remains susceptible to any decisions they make, i.e. throwing toys out of the pram due to charges. These sort of moves have a fairly notable affect on passenger numbers.
Of course, the airfield limitations do not help either, as they significantly reduce the number of airlines who would consider SOU as a viable place to operate from. This is why it is good to see all these new carriers at SOU, as each one makes the airport less influenced by a single, dominant airline.