Two incidents spring to mind.
1. Bad weather.. Departing northbound a/c on south runway.... go-around on north runway. Departure controller instructs departure to fly a heading to maintain separation.... no reply. Tries again.. no reply. A very, very scary minute or two ensued. The departure had changed frequency on getting airborne without being told.
2. Another one during a trial - "Contact xxx (freq) when airborne". Pilot decided to call the centre whilst approaching the holding point. Centre controller assumes it was just airborne and clears it to climb. Pilot assumes this means he can go, which he did. Tower controller totally bemused.
In my book, the controller should decide when a frequency change is appropriate and that is after no possible conflict can exist.