I have come across airports that tell you to turn it on on the ground, and some airports that specifically tell you to turn it off. Some even state the requirement in the ATIS.
By 'it' i mean the transponder. TCAS should always be off on the ground, so you should use the 'auto' position on your box, or 'xponder'.
I guess the safe way would be to use the transponder on the ground, and let ATC tell you to turn it off if it bothers them.
My airline has put a line in the before start checklist and in the shutdown checklist for remembering to turn it on before push-back and off when at the gate.