I reckon in a few months I'm going to be writing the exact same stuff. I was nervous to solo ( still am , and the weather refusing to cooperate means its been months since I did a solo flight) and like you I feel that I'm always overly cautious and I wonder if that is going to become a huge problem for me further along the road.
What was your instructor's take on the conditions? To me it sounds like they backed your decision .
I'm sure someone has said this old saying to you before "it is better to be on the ground wishing you were in the air , than in the air wishing you were on the ground"
it is so true.
No matter what someone else may have done. You absolutely did the right thing for YOU , because you are sitting on the ground trying to justify your decison NOT to carry one, rather than pulling yourself from a pile of wreckage trying to justify the decision that you made TO carry on.
it may be that with more experience you would have flown in those conditions, but you don't have the experience to feel comfortable, so it was the correct thing to do.
Pilot Decison making (PDM) should also be PERSONAL decision making, what is right for one person may not be right for you.
I really hope this helps and I think i understand a little of what you are going through and hope I can take my own advice shortly as well