Difference rounding 10.075 in Excel (10.08) and VB.net (10.07) [duplicate] - excel

The value is the result of dividing the sum of 2.01 and 2.52 by 2 (2.01 + 2.52) / 2. Excel displays this value as 2.265 and when formatted to 2 digits, it's 2.27. However, the value stored in the file is 2.2649999999999997. When I recreate this is C#, I also get that value in my variable, not 2.265. I understand this is due to floating point precision issues with the division of 4.53 by 2.
double result = (2.01 + 2.52) / 2;
Console.WriteLine(result);
The Console displays 2.265 but the value shown in the QuickWatch debugger shows 2.264999999999997. I suspect the conversion of the value to a string on the in the WriteLine method is correcting for the floating point precision error.
When I apply Math.Round(result, 2, MidpointRounding.AwayFromZero), the result comes back as 2.26 not 2.27 as I expected. It seems like it looks at the first number to the right of the digit I want rounded, sees it's a 4 and ignores everything else to the right of it. The problem is that those 9's are only there because of the precision problem and need to be included, or better yet, the value should be 2.265.
What I have done in my code, is to read the text value from the Excel spreadsheet "2.2649999999999997", convert that to a double 2.264999999999999 and then to a string, which gives me "2.265". I then convert that back to a double 2.265 so that I can apply the Math.Round to it and get the expected result of 2.27. Here's the full code:
double result = Convert.ToDouble(((2.01 + 2.52) / 2).ToString());
Console.WriteLine(Math.Round(result, 2, MidpointRounding.AwayFromZero));
Is my approach for floating point precision and rounding issues, relying on ToString to clean it up, the correct one? If not, how should I have done it?

First: The problem is hard. Because 4.53/2 = 2.265. This rounds to 2.27. However the tiniest rounding error in the calculation resulting in a smaller result (2.264999999....) will lead to a rounding to 2.26. This is what is happening here.
To solve this problem you need to have a floating point arithmetic which has the same internal rounding errors as Excel does.
From this document https://en.wikipedia.org/wiki/Numeric_precision_in_Microsoft_Excel it appears as if Excel uses a modified version of IEEE 754, while C# uses IEEE 754. I do not know where the differences are, but it appears as if internally Excel generates different rounding errors.
This document describes the differences: https://support.microsoft.com/en-us/kb/78113/en-us
(For example, Excel does not use denormalized numbers. This implies a different behavior for rounding errors for numbers < 2).
So I assume you cannot solve this using "double"
Update
However, now that I understand that the issue is not the arithmetic, but the way Excel displays the number, maybe this is a solution
Math.Round(Math.Round(result, 3, MidpointRounding.AwayFromZero), 2, MidpointRounding.AwayFromZero)
First round to 3, then to 2 digits. It appears to me that Excel is doing this.

Related

Rounding of decimal number to whole number [duplicate]

I had an application for returning closest matches to certain values in a large cluster of values( as in my earlier question) and I chose a VBA solution. While in the course of using the said application, I observed that the results for the value of 0.5 were not correct. I had been using the VBA Round function which I found to be returning 0 for 0.5 rounded to integer whereas the worksheet round function returns 1. Strangely, the VBA round function returns 2 for 1.5. I had to substitute the worksheet function in place of the VBA one.
Am I missing something?
It's a known issue. The VBA Round() function uses Banker's rounding while the spreadsheet cell function uses arithmetic rounding. Check the details here:
PRB: Round Function different in VBA 6 and Excel Spreadsheet
The workaround proposed by Microsoft is to write a custom function to get the desired results.
Banker's rounding always rounds 0.5 to the nearest even number and is standard in accounting, which is why Excel works that way. Arithmetic rounding rounds 0.5 up to the next number.

Excel rounds up number in one record, rounds down the same number in another

Excel is rounding numbers inconsistently that is causing me issues. When using ROUND(), sometimes it rounds a specific number up, while at other times it rounds the same value down.
I've tried setting Excel to show exact values in settings, but it doesn't change anything.
This is an example of what is happening.
This is the simple formula ROUND((A1-B1)/2,4)
For one record I have the values (.3159 - .3152) which evaluate to .0007 then divide by 2 to get .00035.
For the next record I have the values (.3554 - .3547) which also evaluates to .0007 and divided by 2 results in .00035
So, even though both values are .00035 when I round off to 4 decimal places I am getting .0003 for one and .0004 for another. Same number, rounding to the same number of places, two different results. How can I fix this?
This is an issue with floating point numbers that is inherent and cannot be solved, only avoided.
Try these tests in Excel:
=(0,3159-0,3152)=(0,3554-0,3547) gives you FALSE.
=(0,3159-0,3152)-(0,3554-0,3547) gives you something like 5.55112E-17.
If you cannot accept the differences, you should round already in the middle of the calculation, not only at the end:
=ROUND(0.3159-0.3152,4)=ROUND(0.3554-0.3547,4) is TRUE
=ROUND(0.3159-0.3152,4)-ROUND(0.3554-0.3547,4) is 0
further reading: Is floating point arithmetic stable? and Binary floating point and .NET, by highly regarded Jon Skeet.

Different results when using SUM and not?

Does anyone know why Excel gives different answers to the same question when using SUM function or not?
If you type in:
=0.1+0.1+0.1-0.3
You get a different (correct) response of 0 compared to:
=SUM(0.1+0.1+0.1-0.3)
Which gives an incorrect answer of 5.55112E-17.
I understand that Excel uses the IEEE 754 and that explains why the second is off, but what I would like to know is how the two differ, and what does the first do to get it correct?
This is taken from Microsoft's page explaining floating-point arithmetic:
Example when a value reaches zero
In Excel 95 or earlier, enter the following into a new workbook:
A1: =1.333+1.225-1.333-1.225
Right-click cell A1, and then click Format Cells. On the Number tab,
click Scientific under Category. Set the Decimal places to 15. Instead
of displaying 0, Excel 95 displays -2.22044604925031E-16.
Excel 97, however, introduced an optimization that attempts to correct
for this problem. Should an addition or subtraction operation result
in a value at or very close to zero, Excel 97 and later will
compensate for any error introduced as a result of converting an
operand to and from binary. The example above when performed in Excel
97 and later correctly displays 0 or 0.000000000000000E+00 in
scientific notation.
It appears as though the optimization mentioned in the last paragraph is not applied if brackets are included in the calculation - perhaps it disrupts the calculation sequence. For example:
=0.1+0.1+0.1-0.3 = 0
However:
=(0.1+0.1+0.1-0.3) = 5.551115123125780E-17
Yet, the miscalculation is not only applicable to those numbers in brackets but also the numbers outside, provided there are brackets in the formula. So:
=0.1+0.1+0.1-0.3+(0.1+0.1+0.1-0.3) = 1.110223024625160E-16
This calculation gives twice the error in it's calculation despite the first part not being parenthesised.

Excel if equal and match are not return the same thing

I have put two date time values in cells A1 and B1. The date time is precisely 41703.0416666667. One is ouput from an SQL database the other manually written.
The result of =if(A1=B1,1,0) is 1.
The result of =MATCH(B1,A1,0) is #N/A.
Does anyone have any theories as to why this may be happening?
Probably an issue with converting decimal to binary. See related answer:
Simple HLOOKUP Failing with Excel 2010
This appears to be a limitation of storing floating point numbers in binary - as described here: http://support.microsoft.com/kb/214118
Many combinations of arithmetic operations on floating-point numbers
in Microsoft Excel and Microsoft Works may produce results that appear
to be incorrect by very small amounts. For example, the equation
=1*(.5-.4-.1) may be evaluated to the quantity (-2.78E-17), or -0.0000000000000000278 instead of 0.
This problem is not unique to excel either but rather a result of:
IEEE 754 specifies that numbers be stored in binary format to reduce
storage requirements and allow the built-in binary arithmetic
instructions that are available on all microprocessors to process the
data in a relatively rapid fashion. However, some numbers that are
simple, nonrepeating decimal numbers are converted into repeating
binary numbers that cannot be stored with perfect accuracy.
The issue is with the floating-point calculation (see http://support.microsoft.com/kb/78113), one possible workaround is to work with the round() function. In your case, rounding to 10 or 12 decimals would probably be enough to address the issue.

Excel 2003 not giving me 1 decimal when the

I put an "if" function to give me no decimals if result is >200, if not to give me 1 decimal. It works great, unless the result's decimal is .0, then it doesn't show me my decimal. How can I force the decimal to show?
Here is my function: =IF($E$13>200,ROUND($E$13,0),ROUND($E$13,1))
If representing the number as text is acceptable, the following will solve your problem:
=IF($E$13>200,ROUND($E$13,0),IF(ROUND($E$13,0)=ROUND($E$13,1),CONCATENATE(ROUND($E$13,0),".0"),ROUND($E$13,1)))
If you need to use your number in future calculations, be sure to use the original number, which can be obtained via ROUND($E$13,1).

Resources