Pretty much. Most parts conform to specified dimensions & electrical interfaces. Lots of manufacturers & retailers of equivalent parts so rather more competition than Mac-world.
Buy a case, add a power supply if there wasn't one already in the case (usually there is), buy a motherboard to suit the class of processor you wish to buy, a graphics card if your chosen motherboard hasn't got built in graphics, a HD, CD or DVD & then bolt it all together.
If a part dies then buy a replacement & drop it in. Not always that easy since there may no longer be a processor available that fits the motherboard the dead processor came from. Solution is to buy a new MB & processor.
Upgrades are common. A motherboard will usually be suitable for a range of processors so you can often upgrade after a year or so when the high end processors become cheaper.