Construction cranes are wind speed limited, regardless of being manned or not.
The limit is dependent on the jib length so a longer crane is likely to have a lower limit to a short armed one. The operator has control over when the jib swings free to the extent that they can complete a lift but that time span is quite short. If the limit is exceeded the crane weather vanes once it is unloaded and the hook raised.
Normally cranes are in weather vane mode when not operational, but on some sites 'stops' will be placed to avoid oversail if this causes and problem. Use of stops is in exceptional circumstances only as this imposes structural issues on the tower crane which are not always acceptable.
As others have said using the crane a wind indicators is not a reliable source...during working hours they will be moving unless the wind is too strong, in which case you will already know about it.. out of hours the weather van effect is possible but there are too many variables to make them good guides.