I have a calculation to be done in a query: ((28154/3745181) * 100), whose resultant value is .751739 which I want as 0.75.
But when I try to do it in a query, I get 0 as shown below:
select ((28154/3745181) * 100) --Returns 0 as the result.
How can I get the desired 0.75 as the resultant?
You are doing integer arithmetic because your types are integer types.
select ((cast(28154 as float)/cast(3745181 as float)) * 100)
However you are obviously trying to get percentages so this is an alternative:
select((100 * 28154) / 3745181 )
I.e. do the multiplication first, then the division.
Related
why vba shows answer "0" when Google shows "11.05"?
Sub test2()
Debug.Print 22.15 Mod 11.1
End Sub
Q: Is it possible to get in VBA result the same as Google provide?
UPD2:VBA's Mod operator (not function) differs significantly from Excels MOD function is a few respects. First, it handles negative values differently... second, if you use 1 as the divisor, it will return 0, not the decimal portion of the floating point number... third, it handles floating point number differently (it uses Banker's Rounding to round all floating point number to whole numbers before performing it operation on those numbers whereas Excel doesn't).
As Mod in VBA only deals with integers, you'll have to scale your values, for example with 10 ^ 3 (1000) if you have values of three decimals or less:
DecimalCount = 3
Scaling = 10 ^ DecimalCount
Debug.Print (22.15 * Scaling Mod 11.1 * Scaling) / Scaling
11.05
Debug.Print (22.15 * Scaling Mod 11.075 * Scaling) / Scaling
0
Just adjust DecimalCount to match your expected values.
An alternative to Gustav's answer that doesn't need the scaling: First make an regular division, use the integer part of the result (integer quotient) and subtract the divisor times the (integer) quotient from the dividend.
Function ModDouble(a As Double, b As Double) As Double
Dim y As Double
y = (a / b)
ModDouble = a - (b * Int(y))
End Function
Testing it:
Sub test()
Debug.Print ModDouble(5, 3)
Debug.Print ModDouble(22.15, 11.1)
End Sub
> 2
> 11.05
I've seen multiple answers say the following algorithm works fine to generate a random number between two values. I'm getting spurious results where I'll sometimes get a value returned that is higher than the upper bound.
Dim random as Integer
random = Int (3 - 0 + 1) * Rnd + 0
Debug.Print random
This should give values between 0 - 3 right? I'm seeing 0 to 4 when run a few times..???
This is one of the idiosyncrasies of VBA.
You can see the effect more clearly by writing
random = Int(4) * Rnd
4 * Rnd is a floating point double type, and when it gets assigned to random, The same rounding convention as for CInt is applied1; i.e. if Int(4) * Rnd is 3.5 or greater, the result is 4.
The fix is to write
random = Int(4 * Rnd)
1 The convention "round half to even" is often called Banker's Rounding. See
https://en.wikipedia.org/wiki/Rounding#Round_half_to_even
I think it is to be expected when explicitly converting floating point number to integer, but what might not be expected is that the rounding is towards the closest even number:
Dim i As Integer
i = 3.5 ' i = 4
i = "2.5" ' i = 2
This question already has answers here:
How to get a float result by dividing two integer values using T-SQL?
(10 answers)
Closed 7 years ago.
I am trying to calculate some performance metrics as [RATE] in SQL but no matter what I do I am only getting integer values in return.
In my dataset I have a Numerator and a Denominator both stored as integers that are broken down into groups and subsets. The rate is calculated slightly differently depending on the group and subset. For group 1 the calculation is simply N/D. For group 2 the calculation is (N*1000)/D with the exception of 1 subset which is calculated (N*10000)/D.
I wrote the query as:
SELECT [Group]
,[SubSet]
,[Numerator] N
,[Denominator] D
,CASE WHEN D=0 Then NULL
WHEN [Group]='G1' THEN [N]/[D]
WHEN [SubSet]='S2' THEN ([N]*10000)/[D]
WHEN [SubSet] NOT LIKE 'S2%' AND [G] NOT LIKE 'G1%' THEN ([N]*1000)/[D] as [RATE]
No matter what I do the outcome variables are integers. I tried formatting RATE as varchar, decimal, and float with no success. I tried changing N and D's format to varchar, decimal, and float as well. I tried changing the equations from (N*1000)/D to (N/D)*1000 but still no effect.
What am I missing/doing wrong?
The problem you are having is because SQL is doing integer division, which will only return whole numbers. To get a decimal return value you must have at least one value as a decimal.
Try this:
(CAST([N] as decimal(12,6))/[D]) * 1000
Adjust decimal(12,6) based on the precision you are expecting. 12,6 will return a decimal with 6 digits after the decimal point. If you wanted only 2 decimal places use 16,2.
If you then want to round the calculated value you will need to make use of the ROUND function in SQL.
Round to the second decimal place:
ROUND((CAST([N] as decimal(12,6))/[D]) * 1000, 2)
You need to use CAST:
CAST ((N*1000) AS FLOAT) / D
Hope this helps.
SELECT (n * 1000.0) will do it.
I have user input in two cells, named "UpperRangeHigh" and "UpperRangeLow". I have the following code:
dRangeUpper = [UpperRangeHigh] - [UpperRangeLow]
lLines = Int(dRangeUpper * 100 / lInterval)
The user inputs 120.3 and 120 into the input cells respectively. lInterval has the value 10. VBA produces the result of 2 for lLines, instead of 3.
I can overcome this problem by adding 0.000000001 to dRangeUpper, but I'm wondering if there is a known reason for this behaviour?
This appears to be a problem with Excel's calculation and significant digits. If you do:
=120.3 - 120 and format the cell to display 15 decimal places, the result appears as:
0.2999999999999970
Here is a brief overview which explains how Excel uses binary arithmetic and that this may result in results divergent from what you would expect:
http://excel.tips.net/T008143_Avoiding_Rounding_Errors_in_Formula_Results.html
You can overcome this by forcing a rounded precision, e.g., to 10 decimal places:
lLines = Int(Round(dRangeUpper, 10) * 100 / lInterval
Kindly use single or double when working with decimals to get more accurate results.
Sub sample()
Dim dRangeUpper As Double
dRangeUpper = CDbl("120.3") - CDbl("120")
lLines = Round(CDbl(dRangeUpper * 100 / 10), 4)
End Sub
output = 3
This is a known Floating point issue within Excel
http://support.microsoft.com/kb/78113
From MSDN:
To minimize any effects of floating point arithmetic storage
inaccuracy, use the Round() function to round numbers to the number of
decimal places that is required by your calculation. For example, if
you are working with currency, you would likely round to 2 decimal
places:
=ROUND(1*(0.5-0.4-0.1),2)
In your case, using round() instead of INT should do the trick using 0 rather than 2
I've written a small function in C, which almost do the same work as standart function `fcvt'. As you may know, this function takes a float/double and make a string, representing this number in ANSI characters. Everything works ;-)
For example, for number 1.33334, my function gives me string: "133334" and set up special integer variable `decimal_part', in this example will be 1, which means in decimal part only 1 symbol, everything else is a fraction.
Now I'm curious about what to do standart C function `printf'. It can take %a or %e as format string. Let me cite for %e (link junked):
"double" argument is output in scientific notation
[-]m.nnnnnne+xx
... The exponent always contains two digits.
It said: "The exponent always contains two digits". But what is an Exponent? This is the main question. And also, how to get this 'exponent' from my function above or from `fcvt'.
The notation might be better explained if we expand the e:
[-]m.nnnnnn * (10^xx)
So you have one digit of m (from 0 to 9, but it will only ever be 0 if the entire value is 0), and several digits of n. I guess it might be best to show with examples:
1 = 1.0000 * 10^0 = 1e0
10 = 1.0000 * 10^1 = 1e1
10000 = 1.0000 * 10^4 = 1e4
0.1 = 1.0000 * 10^-1 = 1e-1
1,419 = 1.419 * 10^3 = 1.419e3
0.00000123 = 1.23 * 10^-5 = 1.23e-5
You can look up scientific notation off Google, but it is useful for expressing very large or small numbers like 1232100000000000000 would be 1.2321e24 (I didn't actually count, exponent may be inaccurate).
In C, I think you can actually extract the exponent from the top 12 bits (the first being the sign which you will have to ignore). See: IEEE758-1985 Floating Point
The exponent is the power 10 is raised to then multiplied by the base.
SI is explained at wikipeida. http://en.wikipedia.org/wiki/Scientific_notation
m.nnnnnne+xx is logically equal to m.nnnnnn * 10 ^ +xx
In scientific notation, the exponent is the ten to the XX power, so 1234.5678 can be represented as 1.2345678E03 where the normalized form is multiplied by 10^3 to get the "real" answer.
400 = 4 * 10 ^ 2
2 is the exponent.
If you write a number in scientific notation then the exponent is part of that notation.
You can see a full description here http://en.wikipedia.org/wiki/Scientific_notation, but basically its just another way to write a number, typically used for very large or very small numbers.
Say you have the number 300, that is equal to 3 * 100, or 3 * 10^2 in scientific notation.
If you use %e it will be printed as 3.0e+02