Thanks for the replies so far. It is pretty hard to understand what happened without knowing the exact formulas used by the FMC. The FCOM doesn't provide any details (but that's what the crew in question had available).
Thanks SR71 for your suggestions! I probably need to get myself a Bill Bulfer FMC guide.