I was doing a "percentage change between two numbers" calculation, and I initially wrote it as (using Excel 2013):
=(NEW NUMBER - OLD NUMBER)/OLD NUMBER
I changed it to:
=NEW NUMBER/OLD NUMBER -1
.. to test to see if it was any faster with a large spreadsheet. Like I often do, I checked to see if the generated numbers were all equivalent (using a coerced CellColX = CellColY formula), and about 80% of them were, but others weren't -- by a tiny amount. I watched the action in the Evaluate Formula tool and expanded out the decimal places in the cells to where it was returning all 0's.
For instance, OLD NUMBER = 10.44 and NEW NUMBER = 10.86 resolves to 0.0402298850574713 using the first method, and 0.0402298850574712 using the second method.
However, OLD NUMBER = 13.17 and NEW NUMBER = 10.44 resolves to -0.207289293849658 using either method.
(The OLD and NEW numbers are genuine 2-place decimals.)
Mainly for OCD sake, is there a known reason for this? And is there any significance for very sensitive calculations?
is there a known reason for this?
As explained in Comments, Yes:
As with other spreadsheets, Microsoft Excel works only to limited accuracy because it retains only a certain number of figures to describe numbers (it has limited precision). Excel nominally works with 8-byte numbers by default, a modified 1985 version of the IEEE 754 specification[1] (Besides numbers, Excel uses a few other data types.[2]) Although Excel can display 30 decimal places, its precision for a specified number is confined to 15 significant figures, and calculations may have an accuracy that is even less due to three issues: round off,[3] truncation, and binary storage.
(From Wikipedia.)
is there any significance for very sensitive calculations?
Also Yes, and I suggest not just for very sensitive calculations, rather often where very large or very small numbers are involved in calculations. However because 'very large and 'very small' for normal scales are at the astronomical or anatomic level they are not very often encountered by Excel users. Though there are plenty of examples on SO, eg here and for Google Sheets.
Where these approximations cause problems on a regular basis is, as you have noted, in comparisons. Without rounding results that should be (mathematically) the same may not be and =EXACT or just = may return FALSE falsely.
Related
Regarding Excel 2007 (though it may pertain to other versions):
I want to apply Excel Data Validation to manually inputted data. In this particular case, the input is of the form NN.nnnnh, where the digit "h" is a "half-digit". That is, it can either be 0 or 5.
The spread-sheet converts land-surveying that is manually entered in the form of Feet, Inches, and 16ths of an inch, into decimal feet
The function of the half-digit is to allow the optional higher-precision to 1/32nd of an inch.
For example:
43.0913 is the raw entry for 43 feet, nine inches, and 13/16ths of an inch.
Now, by adding the half-digit in the fifth decimal place, a precision of 1/32" can be expressed.
For example:
27.08135 is the manual entry for 27 feet, 08 inches, and (13.5/16=) 27/32nds of an inch.
The raw input NN.nnnnh is decomposed and converted into feet as a decimal number, using Excel TRUNC function. This manner of conversion is analogous to the more familiar conversion of angles entered as D˚M'S", into DD.dddddd).
I want to assure that the 5th decimal place, manually entered, is ONLY Zero or 5.
I can separately apply logical tests to determine if the fifth-decimal entry is Zero, or 5.
But, when I combine those separate logical tests using the =IF(OR( structure, I get inconsistent results IFF the manually-entered data has an integer value (i.e., in the NN.nnnnh format, any length of just one foot or greater, manually entered as >= 1.00000). Unless I undertake the surveying of table-top architectural scale models, this has serious limitations !!!
I have attached an example spreadsheet to illustrate the formulae used and the results. If anybody can shed some light on this, it would be appreciated.
(If there is a way to simply Attach a *.xlsx example....please let me know. I had intended to do this, and then discovered that it seems to be impossible!)
Use MROUND to test if the number is the same:
=A1=MROUND(A1,0.00005)
I'm attempting to display exactly the first 100 powers of 2 (e.g. 1,2,4,8,16...) in a column in Excel. Unfortunately, once I get up to about 2 raised to the 50th, I'm hit by the Excel limitation of only working with 15 digits - hence, when Excel multiplies 2^49 * 2 I get the wrong answer:
562949953421312.00 * 2 != 1125899906842620.00
I have tried the Excel option "Set Precision as Displayed", but this doesn't seem to help. I suppose it makes sense that there needs to be an upper numeric limit somewhere, but it seems that there should be a way of multiplying large numeric strings, no?
562949953421312 * 2 == 1125899906842624
Excel cannot accurately represent numbers that do not fit into 64 bits. To represent numbers beyond 2^64 (you want to reach 2^100) would require more bits, therefore Excel will lose precision with such numbers.
This is an internal limitation of excel, and you should use a different tool to get exact results. Read the corresponding knowledge base article for more details - there is even a wikipedia page to describe the issues.
Try to wrap the number in the IFERROR function
=IFERROR("1201305162827683","1201305162827683")
We have some weird calculation scenario in Microsoft excel, this simple addition operation resulting 1 at 13 decimal digit whereby it should be zero
But when i extract the value in formula the result is correct (both formula value is the same but the result is different)
From human understanding addition calculation will reduce the decimal digit rather than add it.
Is this by design or bugs ?
I strongly guess that this is a gap from the limited precision of floating point numbers. Accuracy of digital numbers is limited. Numbers in excel are saved in binary format(but displayed in decimal format). This means that the "0" is not as protected as in the decimal system. Usually Excel tries to cover this up for examples like yours.
Also, if your numbers derive from complex calculations (e.g. square-roots), the accuracy can be limited as most functions use approximation with limited iterations to give a result.
You can find more information about floating point arithmetic here. The blog is about pythin but the way it works is similar.
https://docs.python.org/3/tutorial/floatingpoint.html
A colleague of mine sent me their Excel sheet and asked me to take a look at it. The issue is that with a very specific number (56136.598), Excel is automatically extrapolating that number out to 10 decimal places completely regardless of the formatting options.
The cell displays the number to the correct 3 decimal places, but if you look at the number in the formula bar it displays all 10 decimal places. It even changes the number to 10 decimal places if I write the formula =round(56136.598,3) to =round(56136.5979999999,3).
Unfortunately, given the industry I am in, I need some explanation as to why this very specific number induces this change. It's not enough to just use a round or trunc function to lop it off at 3 decimal places, the fact that this number and this cell have a different set up then the rest of the parallel cell calculations is drawing some criticism. Has anyone ran into this before? I have tried it in Excel 2010 and 2019 and in new worksheets, same issue. It seems that excel refuses to accept the number at 3 decimal places and forcing an expansion to 10 decimal places on its own.
This is a normal behavior. See the image below where I just entered 56136,598 into the cell.
This happens due to the fact that Excel is a numeric calculation program and not an algebraic one. So it is a problem of precision. Also see Numeric precision in Microsoft Excel.
Excels results are not absolute but very close to correct. The difference between these to numbers is almost 0 (the difference is 0,0000000001).
And this is actually how most common calculators will act too (you just don't see that). It is just the nature of how calculators (and computers) work.
So there is nothing to worry about.
More about this: Understanding Floating Point Precision, aka “Why does Excel Give Me Seemingly Wrong Answers?”
I have put two date time values in cells A1 and B1. The date time is precisely 41703.0416666667. One is ouput from an SQL database the other manually written.
The result of =if(A1=B1,1,0) is 1.
The result of =MATCH(B1,A1,0) is #N/A.
Does anyone have any theories as to why this may be happening?
Probably an issue with converting decimal to binary. See related answer:
Simple HLOOKUP Failing with Excel 2010
This appears to be a limitation of storing floating point numbers in binary - as described here: http://support.microsoft.com/kb/214118
Many combinations of arithmetic operations on floating-point numbers
in Microsoft Excel and Microsoft Works may produce results that appear
to be incorrect by very small amounts. For example, the equation
=1*(.5-.4-.1) may be evaluated to the quantity (-2.78E-17), or -0.0000000000000000278 instead of 0.
This problem is not unique to excel either but rather a result of:
IEEE 754 specifies that numbers be stored in binary format to reduce
storage requirements and allow the built-in binary arithmetic
instructions that are available on all microprocessors to process the
data in a relatively rapid fashion. However, some numbers that are
simple, nonrepeating decimal numbers are converted into repeating
binary numbers that cannot be stored with perfect accuracy.
The issue is with the floating-point calculation (see http://support.microsoft.com/kb/78113), one possible workaround is to work with the round() function. In your case, rounding to 10 or 12 decimals would probably be enough to address the issue.