Having operated into SFO many times over the years, I would not say it's inherently unsafe provided a crew prepares thoroughly.
Just because an approach is visual does not make it dangerous. But the risk factors need to be addressed for any approach. A good example is an autoland in non LVP conditions. Is it more risky than one flown when LVP are in force?Well, actually it is because of the lack of protection. But provided a crew has strategies to mitigate any threats, then it should be fairly routine.
SFO is a high workload airport for a crew that has perhaps arrived after many long hours of flying. So the detail in the planning and preparation must cover all eventualities including the possibility of a visual approach, sidestep or PRM to name a few examples.
All airline crew are trained to fly a visual approach. It does take more skill than an ILS but it certainly isn't rocket science.
Last but not least, good monitoring from the PM is essential as is his willingness to call out any deviation of correct flight path or SOP.