I use an iPaq 5555 (old but good).
I bought a Fellowes case for it. When it's in the case and the case is shut, it's held completely still and there's no way anything can press any button. Since I've had it in that case (about two years) I've never had a flat battery problem.
It has an SD card in it, and removing and plugging in the card doesn't make it turn on or off.
I keep a spare battery in my "handbag" just in case - but have never needed it yet. The iPaq has a small backup battery - I'm not sure what it does that the main battery doesn't.
In other words: not a problem!