Going back a long while we used to fly one of our HS125 aircraft worldwide with no autopilot at FL410. Never seemed much of an issue except the extra workload. I did turn back once climbing out of Sydney on a 146 with an autopilot failure but that was on a return to the UK ferry.