In a swept wing airliner, one would normally choose a cruise level which provides sufficient margin between high and low speed buffet boundaries. I'm not sure what the current rule is, but 1.35+10 was the normal minimum IAS for the type I flew, although earlier operation allowed 1.2+10 if no turbulence was forecast.
The difficulties of precise hand flying at high level in aircraft with considerable inertia should not be underestimated; one of the early training sorties on the type on which I instructed included a simple 180° level turn at 30° AoB at FL350, so that students (particularly ex-fighter pilots who thought that autopilots were unnecessary) appreciated the problems.
I once had to fly a Vulcan back across the pond at M0.84/FL410 without a serviceable autopilot and it wasn't much fun - we did 20 minutes each at the helm for about 5 hours!
Flying spamcans in Southern UK, my preferred VFR cruising level was 1500' on the nearest aerodrome QNH - I liked the view. Engine failure in an IFR-approved, properly maintained SEP aeroplane was about as likely as being hit by an asteroid, but the risk could generally be mitigated by avoiding flight over hostile terrain or congested areas.