They're a bit old now, & only available second hand, but what about a Toshiba Libretto?
I've only seen one once or twice but was quite impressed with them. They have a VGA width screen but only about half or a third in height size/resolution. Pentium 100 or possibly 133, around a 1 gig HD but that could be upgraded if desired, a PC Card slot & standard ports. They're a standard PC spec - not a Win CE thing.
I wouldn't mind one to complement my PII notebook!