We've been advised by our technical dept that the flight controls can not be damaged by wind. Apparently there is enough dampening effects in the control system to prevent that from happening.
However because the B737 has Manual Reversion (direct cable link between the control wheel/column and ailerons/elevators, it's good to leave the hydraulics pressurised during turn arounds in windy conditions to prevent the control column from moving about between your legs. (The ailerons aren't usually the problem, it's the elevator flapping up and down with tailwinds causing the control column to move backward and forwards that's the problem.)
If it's the last flight of the day and there's a moderate wind blowing from behind the aircraft, I just leave it to the engineers to shut the hydraulics down after they've done their overnight checks. If I was securing the aircraft at a non-maintenance airport and it was windy, I would wait until I was out of my seat before shutting the hydraulics off.