The fear is that you'll go around with them disengaged, but it is not a problem at all.
What's to fear if you're flying the airplane instead of letting it fly you. I really don't understand the problem. And no, I've never flown anything this big. We've had two fatal accidents lately (Amsterdam and Buffalo) where apparently the crews ran out of airspeed by not flying the plane. Looks to me like basic skills have been left on the ground.