I thought the rule was '250kt below FL100 unless in controlled airspace when it can be cancelled by ATC' ie in any controlled airspace you stick to 250kt unless ATC say 'cancel speed restriction'.
Exceptions are military aircraft in Class G and a civil aircraft operating under an exemption from the CAA using the prefix 'Fast' on its callsign and identified on radar.(only has to be identified, not receiving a radar service)