In an earlier life I was involved in "aerial" work, still aviation, albeit with a lot more "leeway". to do your own thing.
We had a guy in the company that was an extremely good pilot, a nice guy and quiet a lot of experience for his age. Unfortunately he was unproductive as he was over cautious. He would probably never have bent an aircraft but was not producing the results per hour, by a long shot. Unfortunately they got rid of him.
It is all about risk management. Knowing what you can get away with and what you cant. This of course only comes with air experience. Experience is not the be all to end all, as it wont stop you getting killed if you go to far but it teaches you a lot about standing up to certain types of commercial pressure (prevalent in GA), for instance. Knowing when to stop flying in certain conditions and when to keep going. However experience is also a great yardstick. Some pilots have the experience but don't seem to mold, for want of a better word.
Of course, we all get scared and the fear of being killed helps keep us alive. A person who has no fear, will kill themselves and others, very quickly I am afraid. Some have more fear than others.