I never say to the Ground Crew "Clear Push...etc" until the brakes are released.
When I have received ATC clearance to push I usually say something like "Standing by on the brakes" to the Ground Crew - I then wait for them to say "Brakes Off", release the brakes and then say "Clear Push".
As has been said above, hand signals are also another cue.