I have recently discovered a weired behaviour of MS Excel as well as Libre Office Calc. When I rounded the following number directly to 1 decimal place (last column), it rounded differently than if I went to two decimal places first (2nd column) and then to one decimal place (3rd column).
number ROUND(A1,2) ROUND(A2,1) ROUND(A1,1)
4.449331525248 4.45 4.50 4.40
I'm not a mathematician, but in my sense the rounding directly to one decimal place (last column) is wrong.
Other numbers this happens to include [3.34690622103941, 4.14677866251134, 3.64939941850228, ...].
Can someone explain this phenomenon to me? Thanks!
(Please move this question to an appropriate community, if necessary)
Your results are different because the function looks at different parts of the number to complete either two or one decimal place for its starting point.
For 2dp it looks at 4.449 which is correctly rounded to 4.45, which then correctly rounds to 1dp as 4.5. This is then an incorrect result for 1dp compared to the original source number due to the extra step and the 9.
The moral is if you need 1dp then do it in one step only 4.449 rounds to 4.4. Based on below 5 goes down, above goes up.
Related
A colleague of mine sent me their Excel sheet and asked me to take a look at it. The issue is that with a very specific number (56136.598), Excel is automatically extrapolating that number out to 10 decimal places completely regardless of the formatting options.
The cell displays the number to the correct 3 decimal places, but if you look at the number in the formula bar it displays all 10 decimal places. It even changes the number to 10 decimal places if I write the formula =round(56136.598,3) to =round(56136.5979999999,3).
Unfortunately, given the industry I am in, I need some explanation as to why this very specific number induces this change. It's not enough to just use a round or trunc function to lop it off at 3 decimal places, the fact that this number and this cell have a different set up then the rest of the parallel cell calculations is drawing some criticism. Has anyone ran into this before? I have tried it in Excel 2010 and 2019 and in new worksheets, same issue. It seems that excel refuses to accept the number at 3 decimal places and forcing an expansion to 10 decimal places on its own.
This is a normal behavior. See the image below where I just entered 56136,598 into the cell.
This happens due to the fact that Excel is a numeric calculation program and not an algebraic one. So it is a problem of precision. Also see Numeric precision in Microsoft Excel.
Excels results are not absolute but very close to correct. The difference between these to numbers is almost 0 (the difference is 0,0000000001).
And this is actually how most common calculators will act too (you just don't see that). It is just the nature of how calculators (and computers) work.
So there is nothing to worry about.
More about this: Understanding Floating Point Precision, aka “Why does Excel Give Me Seemingly Wrong Answers?”
Excel is rounding numbers inconsistently that is causing me issues. When using ROUND(), sometimes it rounds a specific number up, while at other times it rounds the same value down.
I've tried setting Excel to show exact values in settings, but it doesn't change anything.
This is an example of what is happening.
This is the simple formula ROUND((A1-B1)/2,4)
For one record I have the values (.3159 - .3152) which evaluate to .0007 then divide by 2 to get .00035.
For the next record I have the values (.3554 - .3547) which also evaluates to .0007 and divided by 2 results in .00035
So, even though both values are .00035 when I round off to 4 decimal places I am getting .0003 for one and .0004 for another. Same number, rounding to the same number of places, two different results. How can I fix this?
This is an issue with floating point numbers that is inherent and cannot be solved, only avoided.
Try these tests in Excel:
=(0,3159-0,3152)=(0,3554-0,3547) gives you FALSE.
=(0,3159-0,3152)-(0,3554-0,3547) gives you something like 5.55112E-17.
If you cannot accept the differences, you should round already in the middle of the calculation, not only at the end:
=ROUND(0.3159-0.3152,4)=ROUND(0.3554-0.3547,4) is TRUE
=ROUND(0.3159-0.3152,4)-ROUND(0.3554-0.3547,4) is 0
further reading: Is floating point arithmetic stable? and Binary floating point and .NET, by highly regarded Jon Skeet.
I have a large excel file the contains many way-points in Latitude and Longitude in the degree and minutes. My problem is that the numbers can't be rounded and must stay exactly the same, but the last 2 numbers need to be removed (in most cases)
I was wondering if there is a formula that would only allow three characters past the decimal. This is how most my numbers look.
26° 17.82964
However Sometimes they look like this
26° 9.82
I know I can format the cell as a number and set the decimal place to 3, however when I copy and paste it doesn't stay the same.
This formula will truncate (It does not Round) the numbers and give all if less:
=MID(A1,1,FIND(".",A1)+3)
This formula will round, but it will always fill out the numbers to three decimal places (I am aware the OP did not want rounding, this is for others that may want it.):
=LEFT(A1,FIND(" ",A1))&TEXT(ROUND(--MID(A1,FIND(" ",A1),LEN(A1)),3),"0.000")
Using Excel 2013, typing this into two cells yields different results.
=ROUND((10.45-10.00)/1,1) = 0.4
but
=ROUND(0.45/1,1) = 0.5
And I have no idea why this is the case. Anyone have an explanation?
Because Excel uses floating-point math, so results are not guaranteed to be exact. On my Excel 2013 installation, if I enter =10.45-10.00 in a cell, then change that cell to Number with 30 decimal places of precision, the actual answer is 0.449999999999999000000000000000. Since 0.44999... and 0.45 are on opposite sides of the rounding boundary, they round to one decimal place differently.
Edit See also this answer and the question and other answers, which go into much more detail about why Excel's math is inexact.
It looks like the top calculation is creating an infinitesimally small difference. To see this, pull the formula out of the ROUND function. Using the "Increase Decimal" button, expand both values to 15+ decimal places. You'll see that the top value is actually 0.449999999999999.
I have no idea why it actually does this.
I have a excel sheet with a few formulas like this:
A1,A2,A3= 0.13,1.25,2.21
A4: =(A1*A2) =0.16 ( 2 decimal points)
A5: =(A2*A3) =2.76 ( 2 decimal points)
A6: =SUM(A4;A5) =2.93 ( 2 decimal points )
And i want to show 0.16+2.76=2.92
well, there's my problem in bold. i want to add the values from the cells, not the formuls result. How can i do that ? Thank you
Presumably you're working with money which is why you need this.
One way to resolve this is to use =ROUND(A1*A2, 2) etc. and base your subsequent calculations from that.
Do be aware though that you will still occasionally get spurious results due to Excel using a 64 bit IEEE754 floating point double to represent numbers. (Although it does have some extremely clever circumvention techniques - see how it evaluates 1/3 + 1/3 + 1/3 - it will not resolve every possible oddity). If you're building an accounting-style sheet you are best off working in pence, and dividing the final result.
Round the values before you sum, ie:
=ROUND(A1*A2,2)
=ROUND(A2*A3,2)
You could wrap your formulas with the ROUND function:
=ROUND(A1*A2,2)
This will give you 0.16 as opposed to 0.163. Do this for each of your calculations and you'll only be calculating everything to two decimal places. Although I'm not sure why you'd want to do that.