The pilot is responsible for terrain clearance. The only time he is not is when he is being vectored. However I think some controllers have too much faith in our situational awareness. On the other hand, pilots tend to trust a clearance as a safe path.
My opinion is that if you are not flying standard routes, then both you and the controller have to improvise, and for that reason you must be more alert.