Thanks for the input all.
Basically my reasons for keeping the speed below 250 below FL100 are all mentioned.
VFR traffic close to airfields, bird strike risk on the window, fuel economy. Like fireflybob mentioned, a common misconception that high speed saves fuel. Not possible.......
I hear often that "you will get there faster" now I believe the difference between standard speed and high speed would be less then a minute (unless atc requires an early descend). Now, just about one minute earlier on the ground, is that worth the risks mentioned?
As well you might have a ROD which could exceed 3000ft/min within 3000ft on MSA.