I think it is better that the SofS is not ex-military. HH's point is well-made and 2 of the services would always feel themselves to be on the back foot..
Any comparison with the US is invalid. Because UK ministers typically need to serve as MPs first, most of the UK ex-mil who have made it to ministerial office left the services as mid-rankers, largely tactical in experience and without the breadth of joint perspective of a 4* like Mattis (who can be appointed without needing to establish a political career). Also, each of the US services has its own minister and civil servants to fight its corner in any disagreements. By contrast, the UK MoD is effectively a single 'joint' ministry, so it is vital that its ministers are as objective as possible. Single-service 'baggage' is unlikely to be helpful in that regard.
All that said, there must have been a better candidate, surely?