fobs, you can't usually use more than 512MB RAM in Win9.x/Me without a tweak (Vcache chokes up).
See
http://tweakhomepc.virtualave.net/ram/vcache.html & read the sections ">512MB" and ">1GB"
Set MaxFileCache in System.ini to reduce the maximum amount of memory that VCache can use to 512 MB (524,288 KB), or a little less (510MB, or 70% of RAM, whichever is least).
Example:
[VCache]
maxfilecache=522240