Regarding Excel 2007 (though it may pertain to other versions):
I want to apply Excel Data Validation to manually inputted data. In this particular case, the input is of the form NN.nnnnh, where the digit "h" is a "half-digit". That is, it can either be 0 or 5.
The spread-sheet converts land-surveying that is manually entered in the form of Feet, Inches, and 16ths of an inch, into decimal feet
The function of the half-digit is to allow the optional higher-precision to 1/32nd of an inch.
For example:
43.0913 is the raw entry for 43 feet, nine inches, and 13/16ths of an inch.
Now, by adding the half-digit in the fifth decimal place, a precision of 1/32" can be expressed.
For example:
27.08135 is the manual entry for 27 feet, 08 inches, and (13.5/16=) 27/32nds of an inch.
The raw input NN.nnnnh is decomposed and converted into feet as a decimal number, using Excel TRUNC function. This manner of conversion is analogous to the more familiar conversion of angles entered as D˚M'S", into DD.dddddd).
I want to assure that the 5th decimal place, manually entered, is ONLY Zero or 5.
I can separately apply logical tests to determine if the fifth-decimal entry is Zero, or 5.
But, when I combine those separate logical tests using the =IF(OR( structure, I get inconsistent results IFF the manually-entered data has an integer value (i.e., in the NN.nnnnh format, any length of just one foot or greater, manually entered as >= 1.00000). Unless I undertake the surveying of table-top architectural scale models, this has serious limitations !!!
I have attached an example spreadsheet to illustrate the formulae used and the results. If anybody can shed some light on this, it would be appreciated.
(If there is a way to simply Attach a *.xlsx example....please let me know. I had intended to do this, and then discovered that it seems to be impossible!)
Use MROUND to test if the number is the same:
=A1=MROUND(A1,0.00005)
Related
We have some weird calculation scenario in Microsoft excel, this simple addition operation resulting 1 at 13 decimal digit whereby it should be zero
But when i extract the value in formula the result is correct (both formula value is the same but the result is different)
From human understanding addition calculation will reduce the decimal digit rather than add it.
Is this by design or bugs ?
I strongly guess that this is a gap from the limited precision of floating point numbers. Accuracy of digital numbers is limited. Numbers in excel are saved in binary format(but displayed in decimal format). This means that the "0" is not as protected as in the decimal system. Usually Excel tries to cover this up for examples like yours.
Also, if your numbers derive from complex calculations (e.g. square-roots), the accuracy can be limited as most functions use approximation with limited iterations to give a result.
You can find more information about floating point arithmetic here. The blog is about pythin but the way it works is similar.
https://docs.python.org/3/tutorial/floatingpoint.html
A colleague of mine sent me their Excel sheet and asked me to take a look at it. The issue is that with a very specific number (56136.598), Excel is automatically extrapolating that number out to 10 decimal places completely regardless of the formatting options.
The cell displays the number to the correct 3 decimal places, but if you look at the number in the formula bar it displays all 10 decimal places. It even changes the number to 10 decimal places if I write the formula =round(56136.598,3) to =round(56136.5979999999,3).
Unfortunately, given the industry I am in, I need some explanation as to why this very specific number induces this change. It's not enough to just use a round or trunc function to lop it off at 3 decimal places, the fact that this number and this cell have a different set up then the rest of the parallel cell calculations is drawing some criticism. Has anyone ran into this before? I have tried it in Excel 2010 and 2019 and in new worksheets, same issue. It seems that excel refuses to accept the number at 3 decimal places and forcing an expansion to 10 decimal places on its own.
This is a normal behavior. See the image below where I just entered 56136,598 into the cell.
This happens due to the fact that Excel is a numeric calculation program and not an algebraic one. So it is a problem of precision. Also see Numeric precision in Microsoft Excel.
Excels results are not absolute but very close to correct. The difference between these to numbers is almost 0 (the difference is 0,0000000001).
And this is actually how most common calculators will act too (you just don't see that). It is just the nature of how calculators (and computers) work.
So there is nothing to worry about.
More about this: Understanding Floating Point Precision, aka “Why does Excel Give Me Seemingly Wrong Answers?”
I'm creating a simple spreadsheet to calculate some betting odds and keeping track of my wins/losses
when I put fractional odds in one column Excel converts some of them to whole numbers (i.e. the ones that are 1/1, 2/1, etc where it does not do it for odds like 4/11, 7/2, etc.
Is there a way of turning this off?
Please note that some of the top heave fractions (11/2, 11/10 etc) get put into whole numbers such as 5 1/2 etc! And I do not want this to occur either
I've tried the Custom formatting of the cells but all of the denominators will inevitably be different, so having something like ??/28 won't work for me
EDIT:
This was solved using the custom format ??/?? and simply removeing the # that was at the front of the custom cell format dialog box
You simply need to change the cell format; you want to use ???/???. This will make Excel represent any decimal number to the closest fraction approximation it can find using the specified numerator and denominator significant digits (number of ? in the format string)
If the cell input is directly a fraction, it will reduce it if possible but always keeping the fraction format.
Examples:
= .10 will be converted to 1/10
= 0.1231 will be converted to 81/658 (supossing ???/??? format is used).
= 10/100 will be converted to 1/10
= 11/12 will remain as 11/12 as no reduction is possible.
= 1/1 will remain as 1/1
etc.
The behavior you are describing is becuase you are using one of Excel's default fraction formats which are all similar to # ???/??? (take note of the leading #). This format will reduce integral values to the non fractional part.
You could use text format for the cells with the odds, and then the VALUE function in any calculations you need to do with them
So I'm creating a spreadsheet that determines the cost of materials and the number of each material needed in order to complete a desire project using input from myself. Right now the desired project is a wall that is 250x9 that requires replace all the 4x8 sheets of wood with OSB and install Vinyl Siding. The issue I'm running into is I cannot get it to always round up. By that I mean even if the value is 1.1 it should round up. In this specific case I am buying nails for my nail gun in a box of 2,000 and each sheet of OSB will have 32 nails in it. If 250x9 area requires 70.3125 sheets of OSB it means I still have to buy 71 sheets of OSB. If that OSB is 71 sheets then it require that I have 2272 Nails then the result is I need 1.125 Boxes of nails. However I can't seem to get it to show this as 2 boxes because again I still need to purchase more than one box to complete the project. So with that being said if I take the number of OSB needed 70.3125 and I place it in a formula with a roundup function it still rounds down (gives me a headache that there is a roundup and a rounddown function and it will still round down on me. Perhaps it is the way I am using it in the formula that is incorrect, I'm not sure. So let me translate the formula's used and you can let me know if I'm doing something wrong or if there is a function or set of functions that I can use to solve this issue.
=SUM(((B30*C30)+(B35*C35)+(E30*F30)+(H30*I30))/(E9*G9))
This says that if I added Wall1 L*W with Wall2 L*W with Wall3 L*W with Wall4 L*W and divide it by OSB H*W I get the number of sheets needed. Which in this case is 2250/32 basically. But its programmed in a manner that I can input the information for individual walls to different area's and get it to spit out the total SqFt for each wall and give an individual breakdown per wall of material needed with cost associated per sq ft of material bleh bleh bleh. The point is I take the result that is the 70.3125 and I move it to a different workbook and I say "Sheets OSB Needed" and in that box I have
=ROUNDUP(Sheet1!A9,1)
Whereas I'm asking it to roundup A9 which is the result of the above formula by intervals of 1. But the output is still 70 instead of 71. and much the same case with the nails needed. Which can be calculated in a few different manners but regardless the amount of nails needed divided by 2000 would output the decimal answer which yields a value of less than 1.5 and it too provides me with a value of 1 instead of 2 with much the same formula. I could achieve my desired result I suppose with Trunc and Mod functions collaborating using multiple cells to output the different portions of the data. But is there a way to do this that doesn't involve so many cells being used up?
C7
=Trunc(A9)
Removes Decimal from 70.3125
C8
=MOD(A9)
Outputs decimals from 70.3125
C9
=IF(C8<1,"1",C8)
If Decimals are < a whole number make it a whole number
C3
=SUM(C7+C9)
Add the whole number to the Trunc Number to get value desired.
Which I'm already seeing an issue with this if there is no decimals in the sheets needed then wouldn't it always add one because the decimal place would be 0? How can I handle this issue? Isn't there an easier way to do this or a way to code it so that its all nested into one calculation or at least mostly all into one calculation without making a circular reference of some sort?
You need to change the second parameter to a 0 ROUNDUP(70.3125, 1) is 70.3 the 3 must be getting dropped elsewhere or lost in formatting.
ROUNDUP(70.3125, 0) will give 71.
The second parameter of round up is the decimal place. So to round to integers it should be 0 not 1
I was doing a "percentage change between two numbers" calculation, and I initially wrote it as (using Excel 2013):
=(NEW NUMBER - OLD NUMBER)/OLD NUMBER
I changed it to:
=NEW NUMBER/OLD NUMBER -1
.. to test to see if it was any faster with a large spreadsheet. Like I often do, I checked to see if the generated numbers were all equivalent (using a coerced CellColX = CellColY formula), and about 80% of them were, but others weren't -- by a tiny amount. I watched the action in the Evaluate Formula tool and expanded out the decimal places in the cells to where it was returning all 0's.
For instance, OLD NUMBER = 10.44 and NEW NUMBER = 10.86 resolves to 0.0402298850574713 using the first method, and 0.0402298850574712 using the second method.
However, OLD NUMBER = 13.17 and NEW NUMBER = 10.44 resolves to -0.207289293849658 using either method.
(The OLD and NEW numbers are genuine 2-place decimals.)
Mainly for OCD sake, is there a known reason for this? And is there any significance for very sensitive calculations?
is there a known reason for this?
As explained in Comments, Yes:
As with other spreadsheets, Microsoft Excel works only to limited accuracy because it retains only a certain number of figures to describe numbers (it has limited precision). Excel nominally works with 8-byte numbers by default, a modified 1985 version of the IEEE 754 specification[1] (Besides numbers, Excel uses a few other data types.[2]) Although Excel can display 30 decimal places, its precision for a specified number is confined to 15 significant figures, and calculations may have an accuracy that is even less due to three issues: round off,[3] truncation, and binary storage.
(From Wikipedia.)
is there any significance for very sensitive calculations?
Also Yes, and I suggest not just for very sensitive calculations, rather often where very large or very small numbers are involved in calculations. However because 'very large and 'very small' for normal scales are at the astronomical or anatomic level they are not very often encountered by Excel users. Though there are plenty of examples on SO, eg here and for Google Sheets.
Where these approximations cause problems on a regular basis is, as you have noted, in comparisons. Without rounding results that should be (mathematically) the same may not be and =EXACT or just = may return FALSE falsely.