PPRuNe Forums - View Single Post - Excel 2007 multiplication error
View Single Post
Old 28th Sep 2007, 17:33
  #10 (permalink)  
bnt
 
Join Date: Feb 2007
Location: Dublin, Ireland. (No, I just live here.)
Posts: 733
Received 8 Likes on 7 Posts
I've been doing a bit more testing here. There are a lot of combinations of factors that give the same result. I agree that the internal representation of the number is still correct i.e. a floating point number very close to 65535, so all subsequent calculations will be fine. As long as you don't actually need any output, on paper or screen.

BUT - and there's a big but - this affects Filtering too. So, if you are using Filter on results, you might get false positives (too many results) if you filter for 100000, or false negatives (missing results) if you filter on 65535. According to that blog entry, a fix is currently under test.
bnt is offline