Some of the smaller 'desktop' type servers are basically that - a desktop dressed up in server clothes, so whilst they've not been great as a server they might be 'ok' as a desktop.
However, that then puts you in the 'lucky dip' category of finding a desktop server that doesn't sound like an aircraft taking off when you turn it on (as with most servers acoustics come secondary to extreme cooling).
You'd also suffer from being able to find desktop OS based drivers for a server platform.