As ever, it depends what you want to do with it.
Do you want it to be your telephone, too?
Do you want to surf the web via GPRS? If so, do you have a telephone with Bluetooth?
Most of them will play games and MP3s etc; I would do some research into the offerings available and what they will do. The market changes alarmingly rapidly!
I bought an Ipaq 5555 a couple of years ago, second hand, off Ebay. It does everything I ask of it, including browsing the web (BT to GPRS on my phone); it will connect to a WiFi access point if there's one about - but some will ask you to pay for the privilege.
I've got some enormous Excel files on it (such as the entire Churchyard records, with 4500 graves and their locations, occupants, dates and so on; my pilot's logbook; and a bunch of other stuff). It keeps my diary (very well) and my address book, and updates/synchs all of that stuff with the desktop PC and my laptop.
Only catch: I did have to buy some proprietary software for the diary because the standard offering was dreadful (it dumps appointments in certain circumstances, and had me booked to be in two different places 300 miles apart at the same time).
It's not as fast as the latest models, but hey I'm old and slow.
I used to play MP3s on it, but I recently bought myself an Ipod.
If this one popped its clogs, I'd look for another 5555 or a later equivalent.
I looked at similar kit with telephone built in, but they were clunky, in my view. I tried a Blackberry and was underwhelmed. My little Motorola V3 sits in my shirt pocket all day, and talks to the car via BT, the Ipaq via BT, and is still there when I need to make phone calls.
Edit: I bought a GPS unit to plug into it, but the 5555 doesn't have a CF slot and the GPS was CF, so that meant a caddy which made the whole thing a lot bigger. The GPS isn't that good compared with the "dedicated" land-based ones such as TomTom. I tried it with PocketFMS for aviation, and soon gave up on that, too (but we have a GNS430 in the Arrow).
I'm told by those who've tried bluetooth GPS that they aren't quite up to it yet. That may have changed recently.