Actually, for the CH-37 requirements the R-2800-50 was probably not a bad choice. It produced 1900bhp for a 2300lb dry weight, and only required a 14:1 ratio main rotor gearbox. While the XT-38 of similar output weighed just over 700lb, it also would have required additional gear reduction stages that would likely increase engine weight by 25-30%.
The R-2800 at that time was well developed, very reliable, readily available, low cost (compared to a turboshaft like the T-38), and the USN/USMC were familiar with the engine design.
The R-2800-50's BSFC was well under .50 lb/hp-hr, or about 25% lower than a similar turboshaft of that time. The R-2800 engine/drivetrain would be a bit heavier and bulkier, but the T-38 engine/drivetrain would be less efficient and require a larger fuel load for a given mission.