I prefer to go flying instead of thinking of my clothes ...
I guess it is like somebody told me long before, when I was learning how to handle a sailboat: stay away from the ones dressed in marine blazer, white trouser and shiny shoes and go for the others capable of mastering the sea.
I just leave on what I wear anyways, nothing special about piloting (ok, unless on professional duty, but even then casual is my preferred).