Remember that in LVL CHG mode and VNAV SPD the thing controlling your speed is not your engines but your elevator (horizontal stab in fact). This is why I would suggest it is safer than V/S, the consquence of not having enough / having too much thrust in LVL CHG is that you simply don't climb / descend with as much rate. The consequence of not having enough / too much thrust in V/S is that you may stall or overspeed.
A very valid point which is true even when you *do* have an A/T. V/S at high altitude and/or high/low speed can get you into trouble quite quickly, as due to reduced margins in just about everything, it’s easy to make performance demands of the aeroplane that may be outside of its capabilities...