The reason why you need "safe eject" is because Windows can cache data heading for removable devices, meaning that it's not "written through" immediately. If you turn this off (in the Policies for the device) you gain the ability to yank the device out at any time, but at the cost of write performance.
It's not a Windows-specific issue, you have the same thing under UNIX systems - including Mac OS X, which has the bizarre "drag the drive in to the trash" process for ejecting it.