I was originally taught that taxi speed varies exponentially as one's distance from the CFI's office!
It all depends on the situation but I have never seen the point of fast taxi speeds when you are running early. Also sensible taxi speeds give you more thinking time.
As a passenger I have always said I can tell a lot about the pilot by the way in which the a/c is taxied to the holding point.