I learned to fly because I was a terrified airline passenger. I was terrified through some of my training, but I had a great instructor who built my confidence. Then came the solos, some of which were fine and others had me quaking in my boots. I completed all successfully and later picked up both the 'Best PPL' and 'Best IMC rating' awards for the relevant years from my training organisation. I was still very nervous flying alone.
Now, with almost 200 hours, I still have days when flying solo leaves me a little apprehensive, but mostly they don't. Why have I kept going? Determination and a sense of achievement really. Because flying is complex and sometimes a little scary, I wan't to succeed. It is probably good to be a little nervous from time to time - it is your brain working to keep you alert and ready to handle threats to your wellbeing.
If you are worried about coping with harsh conditions, take a look at the 'Flying Wild Alaska' DVDs. They show what small planes can handle (not that I'm recommending you copy them!). Where are you based?
HH