I've done this kind of thing - rebuilt Win 95 over Win 95, but on the advice of Dell I used the "deltree" command at the true DOS Prompt to get rid of the Windows Directory and all its sub-directories. You should check this out first using the "help deltree" command at the Command Prompt.
If you are going to install from the CD you must make sure that the CD Drive will still work after you have cleaned out all the stuff that you don't want. So preserve your Autoexec.bat and Config.sys files and the CD drivers that they load.
As already advised, preserve the stuff you want to keep. Using Deltree will allow you to do a clean install without having to reformat etc.