I think you made the right choice... Mind you I'm very very cautious...
The other day there was a 10kt cross wind, I wanted to practise EFATO, different approaches and then get out in the local area to practise forced landings etc (basically the GASCo list)... The vis and cloud were fine, but for some reason the 10kt crosswind managed to completely put me off. I know the aircrafts limit is 18kt, and well within my capabilities. The airfield I'm at is particularly open and a 10kt wind just seemed particularly concerning... Despite seeing others going out.
Probably wasted a perfectly good flying day/days annual leave when it was flyable... But I wanted less wind for practising those drills... Some may say I' being daft, but I'm still here and will have the opportunity to try another day...