There are a lot of things that can be wrong. Most common mistake is a not correctly installed Cooler.
But if you have a brand like Dell, Compaq this is highly unlikely. The thing is if you build the PC yourself is the cooler might not be correct check the type and brand of cooler, If you have an AMD 2300+ Boxed version with the official version you can forget this.
Also the original cooling stuff between the cooler and your CPU might not do the job seen this on a XP2400+. Just remove the original stuff and buy some white stuff put a little bit on there.
I highly suggest you would not use the silver paste as this is used for overclocking and would contain metal parts that conduct energy to the wrong condensators most retailers/manafucterers do not assume any liability if you use this.
Also check the correct fitting of the coolers as the metal part is a bit lame once in a while (< 0,001%) but it can happen
Last but not least your CPU is F#!cked up don't know why but so once in a year you'd see it. Just return it to your retailer. you should have 1 yr warranty.
If you have a boxed one and the cooler is correctly installed, no bits or chips of the core you'll have 3 yr warranty from AMD they'll replace it, may take a week or 3 or so.
And finally if you have a high end VGA card these need to get rid of their heat as well. Be sure to have an empty PCI Slot next to it.