You have expressed exactly what I was thinking in my original post, and I have about the same number of hours. I cancelled when faced with the same situation, and sure enough there were others at the airfield who would have flown. I can only say I was happy with my decision once the initial disappointment dissipated (and once I saw the actual weather which unfolded at my destination). I'm going to do the trip when the weather permits - the destination is still there.
If you had flown and had a bad experience, you would really have kicked yourself. And your passenger probably took comfort from knowing that you are the sort who is cautious.
At our level of experience, I think this is the sort of decision which becomes more tricky to make and others may have views on whether we worry too much. But, aside from mere disappointment, is there really any downside to erring on the side of caution? I guess it's more difficult if your flying is constrained to weekends. This weather has got to shift soon and we have some bank holiday s coming up, courtesy of HRH.