I worry, all the time. Despite flying helicopters for a living, there are lots of times when I feel really nervous about it for no apparent reason. It gets worse as I get older.
Realise that it's natural. You're taking part in a potentially dangerous activity, and deep down, you know that. Sometimes no amount of rationalising will help, despite telling yourself that loads of things in life are dangerous.
When this is the case, take a couple of deep breaths to help you calm down, and then just feel the fear. It's actually only your mind playing tricks on you. The more you let your mind run you around, the more it will. If you simply feel it and say, "Oh yes, that's fear again", it'll gradually stop having such a hold on you. It's a bit like a kid being followed by shadows in the dark; if you try to avoid the shadows and pretend they're not there they get bigger and more scary; if you confront them you see them for what they are.
Flying is probably slightly more dangerous than some of the other things people have mentioned, looking at it objectively. But that's not why you're scared, or you'd never have started. You've been hiding from a shadow, which has now taken over your life. Have a look at it, and it won't have any hold over you...though thisd may take a little while, as the fear is now a habit.
Good luck; I know how it feels and it's not nice.
