If turning the a/p off makes for a rough flight something's not right. The autopilots do an excellent job but sometimes they're not the answer. Turning the a/t's off makes for a more enjoyable passenger experience, especially on Airbus', on approach. We have a couple of departures, typically with strict noise abatement procedures, where the a/p is required to be on. Otherwise it's a Big Boy philosophy, use automation as appropriate/desired. Most guys can adjust the level of automation to match the demands of a particular flight. VFR, day, just vectors? IMC, terrain, multiple tough restrictions to make, thunderstorms? Guys increase the level of automation.
Raw data? Few guys do it. It's not that tough to ignore the FD's and fly pitch and power until reaching an energy state where that you're hand flying matches the commands of the FD. The lazy man's raw data.