Apple get moaned at, by reviewers, every time they release a product with a non-replaceable battery, yet they keep on doing it.
Well, the counter argument is that in, for example, the Macbook Air, the internal battery saves a large amount of weight (and bulk) by dispensing with the battery housing and associated casing structures. Lots of people do like replaceable batteries in laptops (although the amount of places in which one finds oneself without a nearby plug is shrinking), but lots of people like things smaller, thinner and lighter. It may well be that the former win out in the end, but I presume they've crunched the numbers with their market research.
Arguments I guess are similar for the smaller devices, although things like music players are now SO small that I'd guess a higher percentage of people would be happy to live with a slight bulk increase for more battery options than would be the case with laptops.
Still have an "old" larger iPod from a couple of years ago, and it's still going strong with a decent battery life.
Cheers, P.