Puppy Linux -
http://www.puppylinux.org - is a slick smooth fast mini-distro that's designed to run off a live CD or USB key. Interestingly, it can also run off a rewritable CD/DVD and keep stuff there! Or it can store your personal data on a Windows partition.
I installed it to the harddrive of an old 233 PII with 256Mb memory I have lying around and it's bloody impressive! Very fast, really pretty and very capable!
Lots of variants available, I used PuppyOffice (fat at 120MB download) which has OpenOffice2 included and it really is remarkably good.
Give it a whirl! Standard Puppy is only a 70Mb download.