Have you set the jumpers on the small hard drive to slave? there should be a diagram on the drive that shows the position of the jumpers.
Set the jumpers on your main hard drive to master, again look at the diagram on the hard drive for jumper settings.
Connect the master drive to the end of the ribbon cable and the slave drive to the connector that is part way down the ribbon cable.
Edit: sorry, old drive as master, new drive as slave.