Two things;
When I was trained many moons ago, we were advised to issue an abort instruction only if the takeoff roll was a few seconds old and the aircraft's inertia hadn't had a chance to build up to a significant degree. Admittedly, that was a judgement call, but it made sense to me. If the aircraft had reached a point in its takeoff where you, as an ATCO, couldn't easily determine whether an abort should be issued, simply advise the pilot as clearly as possible of the threat to the aircraft and leave it in his/her hands to decide what to do.
Secondly, if you want an expeditious departure (a) ascertain that the aircraft is ready immediate and (b) when the takeoff clearance is issued, clarify it with "Do not stop on the runway, cleared immediate takeoff!!".