Hard drives tend to go first. An interaction of high speed, delicate moving parts and the expansion/contraction of metal due to heat.
Cheap power supplies go next. They age quickly compared to other hardware because they’re subjected to your home power system (overvoltage, undervoltage, spikes) and because of heat. They produce a lot of heat of their own, and have to contend with all the heat the rest of the computer is producing. Lots of electrical parts have shorter life spans as you subject them to higher operating temperatures.
Pretty much everything else in a computer should last a good 10 years at least. Occasionally an optical drive might have a problem, but I’ve never actually had to replace one because it stopped functioning. At a lower level, well, I’m not an electrical engineer, but I would guess capacitors are most at risk. Overvolt them or get 'em too hot and they’ll bubble and eventually explode through their cap. In a PSU, the risk is from overvoltages and heat, and on the motherboard, just from heat assuming the PSU does its job.
I read a story not too many months ago about a rash of electronics dying after 1-2 years, across all kinds of markets from DVD players to cable boxes and computer monitors. There were some Chinese companies that stole an incomplete formula for the fluid in electrolytic capacitors and were producing and selling them anyway. They turned out to have a much shorter lifespan than typical, and since the producers didn’t know, much shorter than rated as well. I sometimes wonder if one of my monitors has those bad capacitors in it. I can’t turn the monitor off, because turning it on is a 45 minute ordeal of getting capacitors to charge up enough to get the backlight lit. I can hear the high pitch whine.