Seems to me that ReadyBoost is for tightwads who want to use SuperFetch but don't want to buy extra RAM. Personally I'd spend on more RAM and a faster hard drive.
See first para here :
Conclusion : Windows Vista's SuperFetch and ReadyBoost Analyzed
The other half of the equation is one day you or your other half will forget why the SD card is there and pull it out when you want to stick in the SD card from your digicam. Can't do that with RAM.