If there's a fan in the PSU, that should start up when the button is pressed. If it doesn't, I'd bet on the PSU being no longer of this earth.
I've changed far more PSUs than motherboards when friends have said "my PC won't work". (In fact, a ratio of five to zero).
Which reminds me ... I must get another spare PSU to keep in my cupboard. They seem to be getting expensive on eBay these days, though!