I would also suggest that a majority of civil/mil combined airports started out as military airfields, so it is more a case that civilians now use military airfields rather than the military moving into civilian airports.
As you rightly point out, it is economics, and convenience for passengers with the increase in local airports.
Y_G