Who ever told you that private flying is safe? It's not. That's why we spend so much time thinking about what might go wrong, training for it, discussing accidents and considering how we might react, and trying to learn and better prepare ourselves.
If you think flying is safe, then get that out of your head, and start to consider how you'll mitigate the risks.
Some good starts would be:
Train for poor weather, but avoid the poor weather.
Always carry enough fuel for all contingencies, plus a bit extra.
Pay attention to your preflight.
Don't ignore something that seems/looks/sounds different than normal.
Fly as high as possible.
Always have a way out.
That should go a long way to reducing the risks to you.