The key to it is that second order effects remain so, which in GA means the TAS is a lot bigger than the wind.
Obviously if you find a calm day then you can knock it off in a flash. I calibrated my liquid compass on one calm day, but such days are rare.
With the reference to ground objects the wind is no more an issue.
You can reduce the task of calibrating the compass during flight conditions to determine the exact physical heading of the plane and the reading of the compass in the same moment. If you want to get the physical heading from the GPS, you will be affected by the movement of the air (unknown).
Miroc