Difference between cheap and expensive batteries generally is their capacity. Cheaper ones tend to be 1200-1400mAh, while the higher end ones are 2700-4000mAh.
A standard alkaline battery is around 2000mAh, while a Lithium ion battery is 2800mAh. So using a higher capacity rechargeable battery should give you the same, if not longer battery cycle.