"No ATC speed restriction" and it's various other forms does not permit the pilot to;
1. Exceed the speed limits for departure procedures involving turns; or
2. Exceed the mandatory airspace speed limits.
The danger in 1 is that the aircraft leaves the protected airspace of the procedure by exceeding either the standard departure speed limits (not 250Kt!!!!) or lower speeds specified.
The most common example of people not realising the situation is when flying a CAT B aircraft that is a jet eg many of the smaller citations. If I rememebr correctly, CAT B are limited to 165Kt during turns for departure unless something else is specified. Therefore, accelerating to 250Kt before making a turn in a departure procedure may cause the aircraft to proceed outside the limits of protected airspace - terrain or other traffic problems or noise issues. The limit for CAT C is 265Kt so flying 250Kt is not an issue there (unless a lower speed is specified on the procedure of course!!).
If ATC take you off the procedure via a vector then they are responsible for not hitting the hills or the other traffic they separate you from so 250 Kt is not unreasonable.
The danger in 2 is that when one flies above the 250Kt limit there is less time available to see and avoid traffic that ATC are not separating you from ( a legal requirement that ATC can not absolve you from). In Class A, B and C ATC are required to separate IFR from VFR. However, in D, E, F and G they are not. Not everyone interprets class D in the same way - see the way the US operates class D compared to most of Europe.
Therefore, while UK operate class D in a manner more akin to ICAO class C, other places are not the same. Thus in general exceeding 250Kt below FL100 in class D involves extra risk.