Does anyone know why Excel gives different answers to the same question when using SUM function or not?
If you type in:
=0.1+0.1+0.1-0.3
You get a different (correct) response of 0 compared to:
=SUM(0.1+0.1+0.1-0.3)
Which gives an incorrect answer of 5.55112E-17.
I understand that Excel uses the IEEE 754 and that explains why the second is off, but what I would like to know is how the two differ, and what does the first do to get it correct?
This is taken from Microsoft's page explaining floating-point arithmetic:
Example when a value reaches zero
In Excel 95 or earlier, enter the following into a new workbook:
A1: =1.333+1.225-1.333-1.225
Right-click cell A1, and then click Format Cells. On the Number tab,
click Scientific under Category. Set the Decimal places to 15. Instead
of displaying 0, Excel 95 displays -2.22044604925031E-16.
Excel 97, however, introduced an optimization that attempts to correct
for this problem. Should an addition or subtraction operation result
in a value at or very close to zero, Excel 97 and later will
compensate for any error introduced as a result of converting an
operand to and from binary. The example above when performed in Excel
97 and later correctly displays 0 or 0.000000000000000E+00 in
scientific notation.
It appears as though the optimization mentioned in the last paragraph is not applied if brackets are included in the calculation - perhaps it disrupts the calculation sequence. For example:
=0.1+0.1+0.1-0.3 = 0
However:
=(0.1+0.1+0.1-0.3) = 5.551115123125780E-17
Yet, the miscalculation is not only applicable to those numbers in brackets but also the numbers outside, provided there are brackets in the formula. So:
=0.1+0.1+0.1-0.3+(0.1+0.1+0.1-0.3) = 1.110223024625160E-16
This calculation gives twice the error in it's calculation despite the first part not being parenthesised.
Related
The value is the result of dividing the sum of 2.01 and 2.52 by 2 (2.01 + 2.52) / 2. Excel displays this value as 2.265 and when formatted to 2 digits, it's 2.27. However, the value stored in the file is 2.2649999999999997. When I recreate this is C#, I also get that value in my variable, not 2.265. I understand this is due to floating point precision issues with the division of 4.53 by 2.
double result = (2.01 + 2.52) / 2;
Console.WriteLine(result);
The Console displays 2.265 but the value shown in the QuickWatch debugger shows 2.264999999999997. I suspect the conversion of the value to a string on the in the WriteLine method is correcting for the floating point precision error.
When I apply Math.Round(result, 2, MidpointRounding.AwayFromZero), the result comes back as 2.26 not 2.27 as I expected. It seems like it looks at the first number to the right of the digit I want rounded, sees it's a 4 and ignores everything else to the right of it. The problem is that those 9's are only there because of the precision problem and need to be included, or better yet, the value should be 2.265.
What I have done in my code, is to read the text value from the Excel spreadsheet "2.2649999999999997", convert that to a double 2.264999999999999 and then to a string, which gives me "2.265". I then convert that back to a double 2.265 so that I can apply the Math.Round to it and get the expected result of 2.27. Here's the full code:
double result = Convert.ToDouble(((2.01 + 2.52) / 2).ToString());
Console.WriteLine(Math.Round(result, 2, MidpointRounding.AwayFromZero));
Is my approach for floating point precision and rounding issues, relying on ToString to clean it up, the correct one? If not, how should I have done it?
First: The problem is hard. Because 4.53/2 = 2.265. This rounds to 2.27. However the tiniest rounding error in the calculation resulting in a smaller result (2.264999999....) will lead to a rounding to 2.26. This is what is happening here.
To solve this problem you need to have a floating point arithmetic which has the same internal rounding errors as Excel does.
From this document https://en.wikipedia.org/wiki/Numeric_precision_in_Microsoft_Excel it appears as if Excel uses a modified version of IEEE 754, while C# uses IEEE 754. I do not know where the differences are, but it appears as if internally Excel generates different rounding errors.
This document describes the differences: https://support.microsoft.com/en-us/kb/78113/en-us
(For example, Excel does not use denormalized numbers. This implies a different behavior for rounding errors for numbers < 2).
So I assume you cannot solve this using "double"
Update
However, now that I understand that the issue is not the arithmetic, but the way Excel displays the number, maybe this is a solution
Math.Round(Math.Round(result, 3, MidpointRounding.AwayFromZero), 2, MidpointRounding.AwayFromZero)
First round to 3, then to 2 digits. It appears to me that Excel is doing this.
Regarding Excel 2007 (though it may pertain to other versions):
I want to apply Excel Data Validation to manually inputted data. In this particular case, the input is of the form NN.nnnnh, where the digit "h" is a "half-digit". That is, it can either be 0 or 5.
The spread-sheet converts land-surveying that is manually entered in the form of Feet, Inches, and 16ths of an inch, into decimal feet
The function of the half-digit is to allow the optional higher-precision to 1/32nd of an inch.
For example:
43.0913 is the raw entry for 43 feet, nine inches, and 13/16ths of an inch.
Now, by adding the half-digit in the fifth decimal place, a precision of 1/32" can be expressed.
For example:
27.08135 is the manual entry for 27 feet, 08 inches, and (13.5/16=) 27/32nds of an inch.
The raw input NN.nnnnh is decomposed and converted into feet as a decimal number, using Excel TRUNC function. This manner of conversion is analogous to the more familiar conversion of angles entered as D˚M'S", into DD.dddddd).
I want to assure that the 5th decimal place, manually entered, is ONLY Zero or 5.
I can separately apply logical tests to determine if the fifth-decimal entry is Zero, or 5.
But, when I combine those separate logical tests using the =IF(OR( structure, I get inconsistent results IFF the manually-entered data has an integer value (i.e., in the NN.nnnnh format, any length of just one foot or greater, manually entered as >= 1.00000). Unless I undertake the surveying of table-top architectural scale models, this has serious limitations !!!
I have attached an example spreadsheet to illustrate the formulae used and the results. If anybody can shed some light on this, it would be appreciated.
(If there is a way to simply Attach a *.xlsx example....please let me know. I had intended to do this, and then discovered that it seems to be impossible!)
Use MROUND to test if the number is the same:
=A1=MROUND(A1,0.00005)
A colleague of mine sent me their Excel sheet and asked me to take a look at it. The issue is that with a very specific number (56136.598), Excel is automatically extrapolating that number out to 10 decimal places completely regardless of the formatting options.
The cell displays the number to the correct 3 decimal places, but if you look at the number in the formula bar it displays all 10 decimal places. It even changes the number to 10 decimal places if I write the formula =round(56136.598,3) to =round(56136.5979999999,3).
Unfortunately, given the industry I am in, I need some explanation as to why this very specific number induces this change. It's not enough to just use a round or trunc function to lop it off at 3 decimal places, the fact that this number and this cell have a different set up then the rest of the parallel cell calculations is drawing some criticism. Has anyone ran into this before? I have tried it in Excel 2010 and 2019 and in new worksheets, same issue. It seems that excel refuses to accept the number at 3 decimal places and forcing an expansion to 10 decimal places on its own.
This is a normal behavior. See the image below where I just entered 56136,598 into the cell.
This happens due to the fact that Excel is a numeric calculation program and not an algebraic one. So it is a problem of precision. Also see Numeric precision in Microsoft Excel.
Excels results are not absolute but very close to correct. The difference between these to numbers is almost 0 (the difference is 0,0000000001).
And this is actually how most common calculators will act too (you just don't see that). It is just the nature of how calculators (and computers) work.
So there is nothing to worry about.
More about this: Understanding Floating Point Precision, aka “Why does Excel Give Me Seemingly Wrong Answers?”
I have a column of positive and negative numbers, which when summed should balance to zero (it's an accounting sheet).
However, if I use a SUMIF formula, instead of 0, i get:
1.81899E-12 or -9.09495E-13 or similar. (I don't know what this sort of result is called, but I think they represent very large or very small numbers)
I have created a sample document which shows the issue.
It returns a zero if the cell is formatted as a number, but the above result if formatted as general.
I often also find that even the simple SUM function also returns a similar result, as does the SUM in the status bar at the bottom of excel, so it is not just the SUMIF function I am struggling with. However, I have been unable to recreate the issue with the SUM function in my example spreadsheet.
I'm using Excel as part of Home and Business 2013.
Thanks for your help.
As #Dominique pointed out, xxxE-12 is a very, very small number. It is very, very close to zero.
xxxE-12 is Excel's (and most programming languages') way of writing xxx * 10^-12.
As you guessed, this is due to rounding. It however also displays the issues of how computers handles floating-point (decimal) numbers; what you think is 1 / 3 = 0.333 might be represented internally as something like 0.333333681. See https://en.wikipedia.org/wiki/Floating-point_arithmetic, or notably https://en.wikipedia.org/wiki/Floating-point_arithmetic#Accuracy_problems.
Secondly, why this appears if the cell is formatted as "General", but not "Number"? With "Number", you expect an integer part and at most, say, 3 decimals. x.xxE-12 has the largest non-zero component at the 12th (!) decimal. So when displayed, it gets rounded to a nice zero. "General" however attempts to display the number as close to the actual value, which in this case is the xxxE-12.
Also note that this might give you issues if you try to compare your calculated value with zero. Say, =IF(SUMIF(...) = 0, ...; it might not evaluate to TRUE even when you think it does (due to the very small value). The solution is instead to compare the difference of calculated value to zero: =IF(ABS(SUMIF(...) - 0) < 1E-9, ....
Using Excel 2013, typing this into two cells yields different results.
=ROUND((10.45-10.00)/1,1) = 0.4
but
=ROUND(0.45/1,1) = 0.5
And I have no idea why this is the case. Anyone have an explanation?
Because Excel uses floating-point math, so results are not guaranteed to be exact. On my Excel 2013 installation, if I enter =10.45-10.00 in a cell, then change that cell to Number with 30 decimal places of precision, the actual answer is 0.449999999999999000000000000000. Since 0.44999... and 0.45 are on opposite sides of the rounding boundary, they round to one decimal place differently.
Edit See also this answer and the question and other answers, which go into much more detail about why Excel's math is inexact.
It looks like the top calculation is creating an infinitesimally small difference. To see this, pull the formula out of the ROUND function. Using the "Increase Decimal" button, expand both values to 15+ decimal places. You'll see that the top value is actually 0.449999999999999.
I have no idea why it actually does this.