Ah Yes Farmer! Thought about that about a millisecond after I hit the "Post" key! By "Unplug" I mean clicking on the "Safely Remove Hardware" icon in the system tray. Sometimes this simply (gracefully) releases the stick, sometimes it generates a panel and you then have to "Stop" the device. By "Remove" I mean physically unplug the stick.
On my system, you only see the "Safely Remove Hardware" icon when you actually have a memory stick physically plugged in. I have noticed that XP often hides this icon - you have to click on the left arrow to uncover some of the icons.
I expanded my original comments to include External Hard Drives plugged into a USB port. Some of these (usually smaller ones using 2.5" technology) take their power from the USB port (and don't need an external power supply). Others (including the 3.5" ones I am currently using) need more power than is available from the USB port (which is, I think only 100ma) and therefore need power from an adapter.
Going to the Control Panel and clicking on "Eject" does NOT release a memory stick (I think that it only opens the drawer in thee case of a CD or DVD).
Sorry - hope that clears things up?