Having a main flat battery on most modern PDA's be then PalmOS or WM5 powered is not a issue today. As these devices now have backup batteries that maintain.
Also if you intend to use WiFi extensively then choose a device which allows you to swap batteries, as WiFi is very power hungry.
If you're bothered about applications mysterioulsy failing to run, then with the cost of large memory cards (miniSD,SD,CF) plummeting. Just purchase a large card, say 2Gb and either keep the applications installers (CAB files, prc files etc) on this card or as previously suggested use a back up solution to back up the current state to the card. I have found both SpriteBackup for WM5 and BackUpBuddy for PalmOS invaluable.
Michael