Formatting your hard drive should not be a problem if the disk is ok. Make sure you have a Win98 start-up floppy disk which includes an option to boot up with cd rom support. Boot from the floppy, format the c drive, and then clean install windows.
If you want to put a new HD in I would suggest you put it in in place of your current HD as the primary master and disconnect the old one. (You can leave it in situ, just disconnect its data cable). Boot from floppy, run fdisk, format the new disk and then install windows. Once its successfully up and running, re-install all the other software you need. Then re-connect the old drive as the primary slave and you can copy the data files across. Then either leave the old drive as a backup copy or format it and use it as an additional drive.
The old drive won't be a folder off the new drive, it'll be a second HDD so you'll have 2 drives probably called C and D. Your CD Rom will become E etc.
Things to note are that when it boots from floppy, it creates a RAM drive which will probably be D - so even with only one HD in, your CD Rom will be the E drive. As said above you cannot just copy programs across from the old drive, they need to be properly installed from their installation disks onto the new drive.