PPRuNe Forums - View Single Post - Excel 2007 multiplication error
View Single Post
Old 28th September 2007 | 17:33
  #10 (permalink)  
bnt
15 Anniversary
 
Joined: Feb 2007
Posts: 755
Likes: 26
From: Dublin, Ireland. (No, I just live here.)
I've been doing a bit more testing here. There are a lot of combinations of factors that give the same result. I agree that the internal representation of the number is still correct i.e. a floating point number very close to 65535, so all subsequent calculations will be fine. As long as you don't actually need any output, on paper or screen.

BUT - and there's a big but - this affects Filtering too. So, if you are using Filter on results, you might get false positives (too many results) if you filter for 100000, or false negatives (missing results) if you filter on 65535. According to that blog entry, a fix is currently under test.
bnt is offline  
Reply