What causes the wind to vary in direction.

[yawningdog]

Multiple reasons.
As already mentioned, local terrain & building characteristics, relative position of the low/high pressure system to your location and its speed of travel, wind gradient due in part to the Coriolis effect, etc