Here is my excel table:
The cell with 8.83 = =((C8-B8)*24)-D8
*C8 = 4:50PM
*B8 = 7:30AM
*D8 = 0.50
The cell $371.00 = =(E8*B3)
Why does my total show $371.00 when B3 = $42? It should be $370.86. I don't have it set to round but for some reason it keeps on doing it.
Because, the actual result of formula =((C8-B8)*24)-D8 is 8.833333333. Due to cell formatting you are seeing 8.83. If you want result for only two digit after decimal point then use round function like-
=ROUND(((C8-B8)*24)-D8,2)
Then you will get result 370.86. Or you can directly use in resulting cell.
=ROUND(E8,2)*B3
$371 is “technically” the correct amount, mathematically. You are actually doing rounding when you are hand-calculating your cross-check, and that isn’t matching Excel’s unfounded calculation.
( 4:50pm - 7:30am ) is 9.3333333 repeating, or “9-1/3”. Divided by 24 leaves you 8.8333333 repeating, not 8.83. Excel is doing what it’s supposed to do, and 371.00 is the correct amount. If your use case calls for times to be rounded to .01 hours and no further then you’ll need to apply rounding somewhere in cell E8.
This question already has answers here:
VBA rounding problem
(2 answers)
Closed 9 months ago.
I am trying to compare two cells in a table:
The column "MR" is calculated using the formula =ABS([#Value]-A1) to determine the moving range of the column "Value". The values in the "Value" column are not rounded. The highlighted cells in the "MR" column (B3 and B4) are equal. I can enter the formula =B3=B4 into a cell and Excel says that B3 is equal to B4.
But when I compare them in VBA, VBA says that B4 is greater than B3. I can select cell B3 and enter the following into the Immediate Window ? selection.value = selection.offset(1).value. That statement evaluates to false.
I tried removing the absolute value from the formula thinking that might have had something to do with it, but VBA still says they aren't equal.
I tried adding another row where Value=1.78 so MR=0.18. Interestingly, the MR in the new row (B5) is equal to B3, but is not equal to B4.
I then tried increasing the decimal of A4 to match the other values, and now VBA says they are equal. But when I added the absolute value back into the formula, VBA again says they are not equal. I removed the absolute value again and now VBA is saying they are not equal.
Why is VBA telling me the cells are not equal when Excel says they are? How can I reliably handle this situation through VBA going forward?
The problem is that the IEEE 754 Standard for Floating-Point Arithmetic is imprecise by design. Virtually every programming language suffers because of this.
IEEE 754 is an extremely complex topic and when you study it for months and you believe you understand fully, you are simply fooling yourself!
Accurate floating point value comparisons are difficult and error prone. Think long and hard before attempting to compare floating point numbers!
The Excel program gets around the issue by cheating on the application side. VBA on the other hand follows the IEEE 754 spec for Double Precision (binary64) faithfully.
A Double value is represented in memory using 64 bits. These 64 bits are split into three distinct fields that are used in binary scientific notation:
The SIGN bit (1 bit to represent the sign of the value: pos/neg)
The EXPONENT (11 bits, biased in value by +1023)
The MANTISSA (53 bits, 52 bits stored + 1 bit implied)
The mantissa in this system leverages the fact that all binary numbers begin with a digit of 1 and so that 1 is not stored in the bit-pattern. It is implied, increasing the mantissa precision to 53-bits for normal values.
The math works like this: Stored Value = SIGN VALUE * 2^UNBIASED EXPONENT * MANTISSA
Note that a stored value of 1 for the sign bit denotes a negative SIGN VALUE (-1) while a 0 denotes a positive SIGN VALUE (+1). The formula is SIGN VALUE = (-1) ^ (sign bit).
The problem always boils down to the same thing.
The vast majority of real numbers cannot be expressed precisely
within this system which introduces small rounding errors that propagate
like weeds.
It may help to think of this system as a grid of regularly spaced points. The system can represent ONLY the point-values and NONE of the real numbers between the points. All values assigned to a float will be rounded to one of the point-values (usually the closest point, but there are modes that enforce rounding upwards to the next highest point, or rounding downwards). Conducting any calculation on a floating-point value virtually guarantees the resulting value will require rounding.
To accent the obvious, there are an infinite number of real numbers between adjacent representable point-values on this grid; and all of them are rounded to the discreet grid-points.
To make matters worse, the gap size doubles at every Power-of-Two as the grid expands away from true zero (in both directions). For example, the gap length between grid points for values in the range of 2 to 4 is twice as large as it is for values in the range of 1 to 2. When representing values with large enough magnitudes, the grid gap length becomes massive, but closer to true zero, it is miniscule.
With your example numbers...
1.24 is represented with the following binary:
Sign bit = 0
Exponent = 01111111111
Mantissa = 0011110101110000101000111101011100001010001111010111
The Hex pattern over the full 64 bits is precisely: 3FF3D70A3D70A3D7.
The precision is derived exclusively from the 53-bit mantissa and the exact decimal value from the binary is:
0.2399999999999999911182158029987476766109466552734375
In this instance a leading integer of 1 is implied by the hidden bit associated with the mantissa and so the complete decimal value is:
1.2399999999999999911182158029987476766109466552734375
Now notice that this is not precisely 1.24 and that is the entire problem.
Let's examine 1.42:
Sign bit = 0
Exponent = 01111111111
Mantissa = 0110101110000101000111101011100001010001111010111000
The Hex pattern over the full 64 bits is precisely: 3FF6B851EB851EB8.
With the implied 1 the complete decimal value is stored as:
1.4199999999999999289457264239899814128875732421875000
And again, not precisely 1.42.
Now, let's examine 1.6:
Sign bit = 0
Exponent = 01111111111
Mantissa = 1001100110011001100110011001100110011001100110011010
The Hex pattern over the full 64 bits is precisely: 3FF999999999999A.
Notice the repeating binary fraction in this case that is truncated
and rounded when the mantissa bits run out? Obviously 1.6 when
represented in binary base2 can never be precisely accurate in the
same way as 1/3 can never be accurately represented in decimal base10
(0.33333333333333333333333... ≠ 1/3).
With the implied 1 the complete decimal value is stored as:
1.6000000000000000888178419700125232338905334472656250
Not exactly 1.6 but closer than the others!
Now let's subtract the full stored double precision representations:
1.60 - 1.42 = 0.18000000000000015987
1.42 - 1.24 = 0.17999999999999993782
So as you can see, they are not equal at all.
The usual way to work around this is threshold testing, basically an inspection to see if two values are close enough... and that depends on you and your requirements. Be forewarned, effective threshold testing is way harder than it appears at first glance.
Here is a function to help you get started comparing two Double Precision numbers. It handles many situations well but not all because no function can.
Function Roughly(a#, b#, Optional within# = 0.00001) As Boolean
Dim d#, x#, y#, z#
Const TINY# = 1.17549435E-38 'SINGLE_MIN
If a = b Then Roughly = True: Exit Function
x = Abs(a): y = Abs(b): d = Abs(a - b)
If a <> 0# Then
If b <> 0# Then
z = x + y
If z > TINY Then
Roughly = d / z < within
Exit Function
End If
End If
End If
Roughly = d < within * TINY
End Function
The idea here is to have the function return True if the two Doubles are Roughly the same Within a certain margin:
MsgBox Roughly(3.14159, 3.141591) '<---dispays True
The Within margin defaults to 0.00001, but you can pass whatever margin you need.
And while we know that:
MsgBox 1.60 - 1.42 = 1.42 - 1.24 '<---dispays False
Consider the utility of this:
MsgBox Roughly(1.60 - 1.42, 1.42 - 1.24) '<---dispays True
#chris neilsen linked to an interesting Microsoft page about Excel and IEEE 754.
And please read David Goldberg's seminal What Every Computer Scientist Should Know About Floating-Point Arithmetic. It changed the way I understood floating point numbers.
I am trying to get equal result of two exact calculations which are computed in a cell formula and the other one with a UDF:
Function calc()
Dim num as Double
num = 30000000 * ((1 + 8 / 100 / 365) ^ 125)
calc = num
End Function
Result of the calculation is different
A1 = 30000000 * ((1 + 8 / 100 / 365) ^ 125) not equal to A2 = calc()
We can test it with =if(A1=A2, TRUE, FALSE) which is false. I do understand that it has something to do with data types in vba and executing cell formula. Do you know how to make calculations to from vba function(s) and excel cell field(s) to render same result?
So, the calculation in application excel and the calculation in vba are presenting different outputs (what you've presented, with format displaying 20 decimal places):
As such, you would see false when comparing them. You will need to round() or format() to truncate the calculation at a level that is appropriate. E.g.:
calc = round(num,4)
calc = format(num,"0.###0")
The reason this is occurring is because of the inherent math you're using, specifically, ((1 + 8 / 100 / 365) ^ 125), and how that is being truncated/rounded in the allocated memory to each part of the calculation, which differs in VBA and in-application Excel.
Edit: Final image with the VBA changes I'd suggested:
Explanation
Double Data type seems to have flaws being "precise" after the "nth" digit. This is stated as well in the documentation
Precision. When you work with floating-point numbers, remember that they do not always have a precise representation in memory. This could lead to unexpected results from certain operations, such as value comparison and the Mod operator.
Troubleshooting
It seems that is the case here: I set up the value from the division on a cell and the division as formula in another one, although excel interface says there are not differences, when computing that value again, the formula on the sheet seems to be more precise.
Actual result
Further thoughts
It seems that is limited by the data type itself, if precision is not an issue, you may try to round it. If it is critical to be as precise as possible, I would suggest you to connect with an API to something that is able to handle more precision. In this scenario, I would use xlwings to use python.
I'm working on a formula to get the standard deviation. It has been working not until I encountered a zero value which makes the result into #DIV/0!.
This is the screenshot of the expected value.
However, when I used my formula, the Game Time SD returned 0.
How do I exclude it in the calculation if the value in F column is zero? I tried IF(F5:F9 <> 0) but it won't work.
This is the formula I used.
F3 = IFERROR(SUBTOTAL(1,F5:F9),0)
G3 = IFERROR(SUMPRODUCT(SUBTOTAL(2,OFFSET(F5:F9,ROW(F5:F9)-MIN(ROW(F5:F9)),,1))*(G5:G9*F5:F9))/SUBTOTAL(9,F5:F9),0)
H3 = IFERROR(((SUBTOTAL(9,F5:F9)*(SUMPRODUCT(SUBTOTAL(2,OFFSET(F5:F9,ROW(F5:F9)-MIN(ROW(F5:F9)),,1)) *
((H5:H9^2*F5:F9*(F5:F9-1)+(G5:G9*F5:F9)^2)/F5:F9)))-(SUMPRODUCT(SUBTOTAL(9,OFFSET(G5:G9,ROW(G5:G9)-MIN(ROW(G5:G9)),,1)),SUBTOTAL(9,OFFSET(F5:F9,ROW(F5:F9)-MIN(ROW(F5:F9)),,1))))^2)/(SUBTOTAL(9,F5:F9)*(SUBTOTAL(9,F5:F9)-1)))^(1/2),0)
I know the problem is somewhere in F5:F9, since the divisor used is zero
The part you suspected in the code involves dividing by a denominator that happens to be a factor in the numerator. You can avoid a division by zero by simplifying that fraction.
((H5:H9^2*F5:F9*(F5:F9-1)+(G5:G9*F5:F9)^2)/F5:F9)))
can be reduced to
(H5:H9^2*(F5:F9-1) + (G5:G9^2*F5:F9))
Resulting in the formula (3rd line modified)
=IFERROR(((SUBTOTAL(9,F5:F9)*
(SUMPRODUCT(SUBTOTAL(2,OFFSET(F5:F9,ROW(F5:F9)-MIN(ROW(F5:F9)),,1))*
(H5:H9^2*(F5:F9-1) + (G5:G9^2*F5:F9))))-
(SUMPRODUCT(SUBTOTAL(9,OFFSET(G5:G9,ROW(G5:G9)-
MIN(ROW(G5:G9)),,1)),SUBTOTAL(9,OFFSET(F5:F9,ROW(F5:F9)-
MIN(ROW(F5:F9)),,1))))^2)/(SUBTOTAL(9,F5:F9)*
(SUBTOTAL(9,F5:F9)-1)))^(1/2), 0)
In my tests without the enclosing IFERROR, I could set some rows to zero and get values. Only when the square rooted subtotal was negative (which logically should not happen) was the result #NUM.
Hope this helps.
I am using excel and i want to display a value to a certain number of significant figures.
I tried using the following equation
=ROUND(value,sigfigs-1-INT(LOG10(ABS(value))))
with value replaced by the number I am using and sigfigs replaced with the number of significant figures I want.
This formula works sometimes, but other times it doesn't.
For instance, the value 18.036, will change to 18, which has 2 significant figures. The way around this is to change the source formatting to retain 1 decimal place. But that can introduce an extra significant figure. For instance, if the result was 182 and then the decimal place made it change to 182.0, now I would have 4 sig figs instead of 3.
How do I get excel to set the number of sig figs for me so I don't have to figure it out manually?
The formula (A2 contains the value and B2 sigfigs)
=ROUND(A2/10^(INT(LOG10(A2))+1),B2)*10^(INT(LOG10(A2))+1)
may give you the number you want, say, in C2. But if the last digit is zero, then it will not be shown with a General format. You have then to apply a number format specific for that combination (value,sigfigs), and that is via VBA. The following should work. You have to pass three parameters (val,sigd,trg), trg is the target cell to format, where you already have the number you want.
Sub fmt(val As Range, sigd As Range, trg As Range)
Dim fmtstr As String, fmtstrfrac As String
Dim nint As Integer, nfrac As Integer
nint = Int(Log(val) / Log(10)) + 1
nfrac = sigd - nint
If (sigd - nint) > 0 Then
'fmtstrfrac = "." & WorksheetFunction.Rept("0", nfrac)
fmtstrfrac = "." & String(nfrac, "0")
Else
fmtstrfrac = ""
End If
'fmtstr = WorksheetFunction.Rept("0", nint) & fmtstrfrac
fmtstr = String(nint, "0") & fmtstrfrac
trg.NumberFormat = fmtstr
End Sub
If you don't mind having a string instead of a number, then you can get the format string (in, say, D2) as
=REPT("0",INT(LOG10(A2))+1)&IF(B2-(INT(LOG10(A2))+1)>0,"."&REPT("0",B2-(INT(LOG10(A2))+1)),"")
(this replicates the VBA code) and then use (in, say, E2)
=TEXT(C2,D2).
where cell C2 still has the formula above. You may use cell E2 for visualization purposes, and the number obtained in C2 for other math, if needed.
WARNING: crazy-long excel formula ahead
I was also looking to work with significant figures and I was unable to use VBA as the spreadsheets can't support them. I went to this question/answer and many other sites but all the answers don't seem to deal with all numbers all the time. I was interested in the accepted answer and it got close but as soon as my numbers were < 0.1 I got a #value! error. I'm sure I could have fixed it but I was already down a path and just pressed on.
Problem:
I needed to report a variable number of significant figures in positive and negative mode with numbers from 10^-5 to 10^5. Also, according to the client (and to purple math), if a value of 100 was supplied and was accurate to +/- 1 and we wish to present with 3 sig figs the answer should be '100.' so I included that as well.
Solution:
My solution is for an excel formula that returns the text value with required significant figures for positive and negative numbers.
It's long, but appears to generate the correct results according to my testing (outlined below) regardless of number and significant figures requested. I'm sure it can be simplified but that isn't currently in scope. If anyone wants to suggest a simplification, please leave me a comment!
=TEXT(IF(A1<0,"-","")&LEFT(TEXT(ABS(A1),"0."&REPT("0",sigfigs-1)&"E+00"),sigfigs+1)*10^FLOOR(LOG10(TEXT(ABS(A1),"0."&REPT("0",sigfigs-1)&"E+00")),1),(""&(IF(OR(AND(FLOOR(LOG10(TEXT(ABS(A1),"0."&REPT("0",sigfigs-1)&"E+00")),1)+1=sigfigs,RIGHT(LEFT(TEXT(ABS(A1),"0."&REPT("0",sigfigs-1)&"E+00"),sigfigs+1)*10^FLOOR(LOG10(TEXT(ABS(A1),"0."&REPT("0",sigfigs-1)&"E+00")),1),1)="0"),LOG10(TEXT(ABS(A1),"0."&REPT("0",sigfigs-1)&"E+00"))<=sigfigs-1),"0.","#")&REPT("0",IF(sigfigs-1-(FLOOR(LOG10(TEXT(ABS(A1),"0."&REPT("0",sigfigs-1)&"E+00")),1))>0,sigfigs-1-(FLOOR(LOG10(TEXT(ABS(A1),"0."&REPT("0",sigfigs-1)&"E+00")),1)),0)))))
Note: I have a named range called "sigfigs" and my numbers start in cell A1
Test Results:
I've tested it against the wikipedia list of examples and my own examples so far in positive and negative. I've also tested with a few values that gave me issues early on and all seem to produce the correct results.
I've also tested with a few values that gave me issues early on and all seem to produce the correct results now.
3 Sig Figs Test
99.99 -> 100.
99.9 -> 99.9
100 -> 100.
101 -> 101
Notes:
Treating Negative Numbers
To Treat Negative Numbers, I have included a concatenation with a negative sign if less than 0 and use the absolute value for all other work.
Method of construction:
It was initially divided into about 6 columns in excel that performed the various steps and at the end I merged all of the steps into one formula above.
Use scientific notation, say if you have 180000 and you need 4 sigfigs the only way is to type as 1.800x10^5
I added to your formula so it also automatically displays the correct number of decimal places. In the formula below, replace the digit "2" with the number of decimal places that you want, which means you would need to make four replacements. Here is the updated formula:
=TEXT(ROUND(A1,2-1-INT(LOG10(ABS(A1)))),"0"&IF(INT(LOG10(ABS(ROUND(A1,2-1-INT(LOG10(ABS(A1)))))))<1,"."&REPT("0",2-1-INT(LOG10(ABS(ROUND(A1,2-1-INT(LOG10(ABS(A1)))))))),""))
For example, if cell A1 had the value =1/3000, which is 0.000333333.., the above formula as-written outputs 0.00033.
This is an old question, but I've modified sancho.s' VBA code so that it's a function that takes two arguments: 1) the number you want to display with appropriate sig figs (val), and 2) the number of sig figs (sigd). You can save this as an add-in function in excel for use as a normal function:
Public Function sigFig(val As Range, sigd As Range)
Dim nint As Integer
Dim nfrac As Integer
Dim raisedPower As Double
Dim roundVal As Double
Dim fmtstr As String
Dim fmtstrfrac As String
nint = Int(Log(val) / Log(10)) + 1
nfrac = sigd - nint
raisedPower = 10 ^ (nint)
roundVal = Round(val / raisedPower, sigd) * raisedPower
If (sigd - nint) > 0 Then
fmtstrfrac = "." & String(nfrac, "0")
Else
fmtstrfrac = ""
End If
If nint <= 0 Then
fmtstr = String(1, "0") & fmtstrfrac
Else
fmtstr = String(nint, "0") & fmtstrfrac
End If
sigFig = Format(roundVal, fmtstr)
End Function
It seems to work in all the use cases I've tried so far.
Rounding to significant digits is one thing... addressed above. Formatting to a specific number of digits is another... and I'll post it here for those of you trying to do what I was and ended up here (as I will likely do again in the future)...
Example to display four digits:
.
Use Home > Styles > Conditional Formatting
New Rule > Format only cells that contain
Cell Value > between > -10 > 10 > Format Number 3 decimal places
New Rule > Format only cells that contain
Cell Value > between > -100 > 100 > Format Number 2 decimal places
New Rule > Format only cells that contain
Cell Value > between > -1000 > 1000 > Format Number 1 decimal place
New Rule > Format only cells that contain
Cell Value > not between > -1000 > 1000 > Format Number 0 decimal places
.
Be sure these are in this order and check all of the "Stop If True" boxes.
The formula below works fine. The number of significant figures is set in the first text formula. 0.00 and 4 for 3sf, 0.0 and 3 for 2sf, 0.0000 and 6 for 5sf, etc.
=(LEFT((TEXT(A1,"0.00E+000")),4))*POWER(10,
(RIGHT((TEXT(A1,"0.00E+000")),4)))
The formula is valid for E+/-999, if you have a number beyond this increase the number of the last three zeros, and change the second 4 to the number of zeros +1.
Note that the values displayed are rounded to the significant figures, and should by used for display/output only. If you are doing further calcs, use the original value in A1 to avoid propagating minor errors.
As a very simple display measure, without having to use the rounding function, you can simply change the format of the number and remove 3 significant figures by adding a decimal point after the number.
I.e. #,###. would show the numbers in thousands. #,###.. shows the numbers in millions.
Hope this helps
You could try custom formatting instead.
Here's a crash course: https://support.office.com/en-nz/article/Create-a-custom-number-format-78f2a361-936b-4c03-8772-09fab54be7f4?ui=en-US&rs=en-NZ&ad=NZ.
For three significant figures, I type this in the custom type box:
[>100]##.0;[<=100]#,##0
You could try
=ROUND(value,sigfigs-(1+INT(LOG10(ABS(value)))))
value :: The number you wish to round.
sigfigs :: The number of significant figures you want to round to.