From memory the word "to" in a clearance to climb or descend was dropped as a result of a Flying Tiger freighter accident on landing in Kuala Lumpur in the late 80's.
I think it was more the case that four digits for an altitude (e.g. "descend
two four zero zero" in this case) was eventually changed to "descend to
two thousand four hundred".
As previous posters have pointed out, "TO" is still correct phraseology and hasn't been dropped.
The Flying Tigers F/O did not have an approach chart available where 2400' on it would have been patently obvious and may have saved the day despite the "descend
two four zero zero" transmission.