Every computer that I have had has had a memory upgrade which has immediately boosted its performance.
Yes, you can probably manage your system so that it is not using the HDD for swap files but you will spend more time opening and closing applications. There are lots of programs that run services in the background and more RAM can only help.
I have a netbook that came with 1Gb on non-upgradeable RAM. I upgraded with a further 1Gb and it ran far sweeter. It seems the original RAM is not in a user accessible DIMM slot but the second slot was the same as on the laptop.