Just dress normally; you're going to an office to get an official document, so don't treat it as if you're about to hit the beach, but there is really no need to go crazy. To be honest, I think I may have gone in Sandals/Shorts/T-Shirt; or maybe it was a polo-neck. Anyway, no-one blinked an eye, and the offocial and I had a very pleasant conversation about England and how much he enjoyed his visits. If you know how to behave in a social situation, as everyone should, it really is a non-event.
As for not flying in Shorts/T-Shirt? Rubbish! Everyone in Florida does, even the instructors at my (very high quality) flight school when I was doing the IR/CPL this year were in shorts and a polo shirt! It's pure practicality.
I've flown extensively in both Florida and California. I prefer Florida for training, and California (and the surrounding states) for touring with a tent.