I have this query that calculates the sum, but it gives me decimal numbers, how can I get the decimal?
Cursor cursor = dataBase.rawQuery(
"SELECT ROUND(SUM(ore),2) AS totore FROM "+DbHelper.TURNI_TABLE+" WHERE MESE = 'Gennaio'",null);
SQLite does not have a decimal data type with a fixed number of digits.
ROUND returns a plain floating-point number, i.e., ROUND(42) will return 42.0; it is your program's responsibility to format this number as you want.
If the only reason you're using ROUND is that you want to format the number, you should not do this in the database but only in your program.
Related
I have an excel file with the value 6228480018362050000 the exported csv looks like this...
Int,Bigint,String
1,6228480018362050000,Very big
When I try running the following code...
InputStream inp = new FileInputStream("/.../test.xlsx");
DataFormatter df = new DataFormatter(true);
df.formatCellValue(WorkbookFactory.create(inp).getSheetAt(0).getRow(1).getCell(1));
I get 6228480018362049500 which is the wrong number because precision is hosed. Is there a way to get the actual value?
If we put long numbers into Excel cells, then those numbers will be truncated to 15 significant digits. This is because Excel does not know such things like big integers. It has only floating point to store numeric values. And with those it follows the IEEE 754 specification. But some numbers cannot be stored as floating point numbers according to the IEEE 754 specification. With your example the 6228480018362050000, which is 6.22848001836205E+018, cannot be stored as such. It will be 6.2284800183620495E+018 or 6228480018362049500 according to IEEE 754 specification.
Microsoft's knowledge base mentions: "Excel follows the IEEE 754 specification on how to store and calculate floating-point numbers. Excel therefore stores only 15 significant digits in a number, and changes digits after the fifteenth place to zeroes."
This is not the whole truth. In reality at least with Office OpenXML (*.xlsx) it stores the values according to IEEE 754 specification and not only 15 significant digits. With your example it stores <v>6.2284800183620495E+18</v>. But thats secondary. Because even if it would store 6.22848001836205E+018, somewhere this must be reconverted to floating point and then it will be 6.2284800183620495E+18 again. Excel does the same while opening the workbook. It converts <v>6.2284800183620495E+18</v> to floating point and then it only displays 15 significant digits.
So if you really need to store the 6228480018362050000 as a number in Excel, then the only way to get the same results as in Excel is to do the same as Excel. To do so we can use BigDecimal and it's round method which is able to use a MathContext with setted precision.
Example:
import org.apache.poi.ss.usermodel.*;
import java.io.*;
import java.math.BigDecimal;
import java.math.MathContext;
class ReadExcelBigNumbers {
public static void main(String[] args) throws Exception{
for (int i = 0; i < 10; i++) {
String v = "6.2284800183620" + i + "E+018";
double d = Double.parseDouble(v);
System.out.print(v + "\t");
System.out.print(d + "\t");
BigDecimal bd = new BigDecimal(d);
v = bd.round(new MathContext(15)).toPlainString();
System.out.println(v);
}
InputStream inp = new FileInputStream("test.xlsx");
Workbook wb = WorkbookFactory.create(inp);
for (int i = 1; i < 9; i++) {
double d = wb.getSheetAt(0).getRow(i).getCell(1).getNumericCellValue();
BigDecimal bd = new BigDecimal(d);
String v = bd.round(new MathContext(15)).toPlainString();
System.out.println(v);
}
}
}
The first part prints:
6.22848001836200E+018 6.2284800183620004E18 6228480018362000000
6.22848001836201E+018 6.2284800183620096E18 6228480018362010000
6.22848001836202E+018 6.2284800183620198E18 6228480018362020000
6.22848001836203E+018 6.2284800183620301E18 6228480018362030000
6.22848001836204E+018 6.2284800183620403E18 6228480018362040000
6.22848001836205E+018 6.2284800183620495E18 6228480018362050000
6.22848001836206E+018 6.2284800183620598E18 6228480018362060000
6.22848001836207E+018 6.22848001836207E18 6228480018362070000
6.22848001836208E+018 6.2284800183620803E18 6228480018362080000
6.22848001836209E+018 6.2284800183620905E18 6228480018362090000
There you can see the difference between wanted floating point value, real floating point value according IEEE 754 specification and reformatted BigDecimal. As you see only the 6.22848001836207E+018 can be stored according to the IEEE 754 specification directly.
The second part does the same using the following Excel sheet:
Another possible workaround is mentioned in the knowledge base article : "To work around this behavior, format the cell as text, then type the numbers. The cell can then display up to 1,024 characters. ". This is good if the numbers are not really numbers but Identifiers for example or some other strings where the digits are only meant as characters. Calculations with such "Text-Numbers" are of course not possible without reconverting them to floating point which will bring the problem again.
There is no change (loss or gain) of precision between 6228480018362050000 and 6228480018362049500. They are simply two different decimal presentations of the same internal binary value, which in decimal is exactly 6228480018362049536, by the way.
Regardless of the cell format, Excel displays (not "stores") only up to the first 15 significant digits, rounding any digits to the right [1].
However, other applications and file formats show up to the first 17 significant digits (or more), which is really what the IEEE 754 standard requires in order to represent every binary value [2]. Apparently, that is true of Apache POI and OpenXML.
You can demonstrate this by doing the following.
In Excel, enter 6228480018362050000. Save as XML.
Open the XML file in Notepad. Note that the Cell/Data element shows 6.2284800183620495E+18, which is 6228480018362049500.
Open the XML file in Excel. Note that Excel still displays 6228480018362050000 in the Formula Bar and in the cell formatted as Number.
It is true that Excel truncates manually-entered numbers (including those read from CSV and TXT files) to the first 15 significant digits, replacing any digits to the right with zeros. But Excel VBA does not.
So for another demonstration, enter the following in VBA, then execute the procedure.
Sub doit()
Range("a1:a2").NumberFormat = "0"
Range("a1") = CDbl("6228480018362050000")
Range("a2") = CDbl("6228480018362049536")
Columns("a").AutoFit
Range("b2") = "=match(a1,a2,0)"
End Sub
Note that A1 and A2 display 6228480018362050000. B2 displays 1, indicating that the internal binary values are an exact match, and VBA does not truncate after the first 15 significant digits.
Explanation....
Excel and most applications use IEEE 754 double-precision to represent numeric values. The binary representation is the sum of 53 consecutive powers of 2 ("bits") times an exponential factor.
Consequently, only integers up to 9007199254740992 (2^53) can be represented exactly. (But note that Excel displays 9007199254740990 for =2^53 because of its 15-significant-digit formatting limitation.)
Most larger integers can only be approximated.
And that is true of most decimal fractions as well, regardless of the number of significant digits. That is part of the reason why =10.1-10 displays 0.0999999999999996 in the Formula Bar and in the cell formatted with 16 decimal places (15 significant digits).
But beware: a calculated value that displays as 6228480018362050000 might differ from the actual internal binary value.
For example, if you enter 6228480018362050000 into A1 and the formula =6228480018362050000+1600 into A2, both A1 and A2 display 6228480018362050000.
But =MATCH(A1,A2,0) returns #N/A, which indicates that the internal binary values are not an exact match.
And the XML file would show 6.2284800183620516E+18 in the Data element corresponding to the Cell element for A2, which is 6228480018362051600. The actual internal binary value, in decimal, is exactly 6228480018362051584.
(FYI, the Excel equal operator ("=") does not compare the internal binary values. Instead, it compares the values rounded to 15 significant digits. So =(A1=A2) returns TRUE misleadingly. It is intended to be a feature; but it is implemented inconsistently.)
If you copy A2 and paste-value into A3, =MATCH(A1,A3,0) continues to return #N/A. But if you subsequently "edit" A3 (e.g. press f2, then Enter), =MATCH(A1,A3,0) returns 1. The internal value of A3 has been changed to the binary representation of 6228480018362050000.
I wonder if that is actually the mysterious problem that you encountered, and you inadvertently oversimplified it with your example.
Does that help?
[1] Cell format does not affect the internal binary value with two exceptions: (1) when Precision As Displayed is set, which is almost never recommended; and (2) when the cell value is calculated, and the worksheet is saved in CSV or TXT file, then re-open or imported in Excel.
[2] Although IEEE 754 specifies that 17 significant decimal digits are the minimum needed to represent all binary values, that does not mean that only 17 significant decimal digits are "stored". As demonstrated above, 6228480018362049500 is actually stored as exactly 6228480018362049536.
This question already has answers here:
How to get a float result by dividing two integer values using T-SQL?
(10 answers)
Closed 7 years ago.
I am trying to calculate some performance metrics as [RATE] in SQL but no matter what I do I am only getting integer values in return.
In my dataset I have a Numerator and a Denominator both stored as integers that are broken down into groups and subsets. The rate is calculated slightly differently depending on the group and subset. For group 1 the calculation is simply N/D. For group 2 the calculation is (N*1000)/D with the exception of 1 subset which is calculated (N*10000)/D.
I wrote the query as:
SELECT [Group]
,[SubSet]
,[Numerator] N
,[Denominator] D
,CASE WHEN D=0 Then NULL
WHEN [Group]='G1' THEN [N]/[D]
WHEN [SubSet]='S2' THEN ([N]*10000)/[D]
WHEN [SubSet] NOT LIKE 'S2%' AND [G] NOT LIKE 'G1%' THEN ([N]*1000)/[D] as [RATE]
No matter what I do the outcome variables are integers. I tried formatting RATE as varchar, decimal, and float with no success. I tried changing N and D's format to varchar, decimal, and float as well. I tried changing the equations from (N*1000)/D to (N/D)*1000 but still no effect.
What am I missing/doing wrong?
The problem you are having is because SQL is doing integer division, which will only return whole numbers. To get a decimal return value you must have at least one value as a decimal.
Try this:
(CAST([N] as decimal(12,6))/[D]) * 1000
Adjust decimal(12,6) based on the precision you are expecting. 12,6 will return a decimal with 6 digits after the decimal point. If you wanted only 2 decimal places use 16,2.
If you then want to round the calculated value you will need to make use of the ROUND function in SQL.
Round to the second decimal place:
ROUND((CAST([N] as decimal(12,6))/[D]) * 1000, 2)
You need to use CAST:
CAST ((N*1000) AS FLOAT) / D
Hope this helps.
SELECT (n * 1000.0) will do it.
Can MS Excel do rounding but only up to the nearest thousandths place and ignore the rest of the decimals in a formula? I've tried value, fixed, round and none of these do what I need.
Let's say I have 100 square feet of space and I pay $1.00566399 per sq foot, I need the formula to round only up to the 5 and ignore the rest of the numbers because when you speak on longer terms it makes a difference in rate.
So I should be multiplying 100sf by $1.01 = $101.00 and not get 100sf * 1.00566399 = $101.57
=trunc(A1,5)
If you want to round up, maybe something like
=trunc((A1*10000+1)/10000,5)
Use the TRUNC($cellRef or number, Decimal places) function. It reduces the number of decimal places WITHOUT rounding, then do your math.
So for example:
=TRUNC(1.00566399,3)
=A1*ROUNDUP(B1,2)
Where A1 contains the number of square feet and B1 contains the price per square foot in it's original long decimal form.
Suppose I want to conver the number 0.011124325465476454 to string in MATLAB.
If I hit
mat2str(0.011124325465476454,100)
I get 0.011124325465476453 which differs in the last digit.
If I hit num2str(0.011124325465476454,'%5.25f')
I get 0.0111243254654764530000000
which is padded with undesirable zeros and differs in the last digit (3 should be 4).
I need a way to convert numerics with random number of decimals to their EXACT string matches (no zeros padded, no final digit modification).
Is there such as way?
EDIT: Since I din't have in mind the info about precision that Amro and nrz provided, I am adding some more additional info about the problem. The numbers I actually need to convert come from a C++ program that outputs them to a txt file and they are all of the C++ double type. [NOTE: The part that inputs the numbers from the txt file to MATLAB is not coded by me and I'm actually not allowed to modify it to keep the numbers as strings without converting them to numerics. I only have access to this code's "output" which is the numerics I'd like to convert]. So far I haven't gotten numbers with more than 17 decimals (NOTE: consequently the example provided above, with 18 decimals, is not very indicative).
Now, if the number has 15 digits eg 0.280783055069002
then num2str(0.280783055069002,'%5.17f') or mat2str(0.280783055069002,17) returns
0.28078305506900197
which is not the exact number (see last digits).
But if I hit mat2str(0.280783055069002,15) I get
0.280783055069002 which is correct!!!
Probably there a million ways to "code around" the problem (eg create a routine that does the conversion), but isn't there some way using the standard built-in MATLAB's to get desirable results when I input a number with random number of decimals (but no more than 17);
My HPF toolbox also allows you to work with an arbitrary precision of numbers in MATLAB.
In MATLAB, try this:
>> format long g
>> x = 0.280783054
x =
0.280783054
As you can see, MATLAB writes it out with the digits you have posed. But how does MATLAB really "feel" about that number? What does it store internally? See what sprintf says:
>> sprintf('%.60f',x)
ans =
0.280783053999999976380053112734458409249782562255859375000000
And this is what HPF sees, when it tries to extract that number from the double:
>> hpf(x,60)
ans =
0.280783053999999976380053112734458409249782562255859375000000
The fact is, almost all decimal numbers are NOT representable exactly in floating point arithmetic as a double. (0.5 or 0.375 are exceptions to that rule, for obvious reasons.)
However, when stored in a decimal form with 18 digits, we see that HPF did not need to store the number as a binary approximation to the decimal form.
x = hpf('0.280783054',[18 0])
x =
0.280783054
>> x.mantissa
ans =
2 8 0 7 8 3 0 5 4 0 0 0 0 0 0 0 0 0
What niels does not appreciate is that decimal numbers are not stored in decimal form as a double. For example what does 0.1 look like internally?
>> sprintf('%.60f',0.1)
ans =
0.100000000000000005551115123125782702118158340454101562500000
As you see, matlab does not store it as 0.1. In fact, matlab stores 0.1 as a binary number, here in effect...
1/16 + 1/32 + 1/256 + 1/512 + 1/4096 + 1/8192 + 1/65536 + ...
or if you prefer
2^-4 + 2^-5 + 2^-8 + 2^-9 + 2^-12 + 2^13 + 2^-16 + ...
To represent 0.1 exactly, this would take infinitely many such terms since 0.1 is a repeating number in binary. MATLAB stops at 52 bits. Just like 2/3 = 0.6666666666... as a decimal, 0.1 is stored only as an approximation as a double.
This is why your problem really is completely about precision and the binary form that a double comprises.
As a final edit after chat...
The point is that MATLAB uses a double to represent a number. So it will take in a number with up to 15 decimal digits and be able to spew them out with the proper format setting.
>> format long g
>> eps
ans =
2.22044604925031e-16
So for example...
>> x = 1.23456789012345
x =
1.23456789012345
And we see that MATLAB has gotten it right. But now add one more digit to the end.
>> x = 1.234567890123456
x =
1.23456789012346
In its full glory, look at x, as MATLAB sees it:
>> sprintf('%.60f',x)
ans =
1.234567890123456024298320699017494916915893554687500000000000
So always beware the last digit of any floating point number. MATLAB will try to round things intelligently, but 15 digits is just on the edge of where you are safe.
Is it necessary to use a tool like HPF or MP to solve such a problem? No, as long as you recognize the limitations of a double. However tools that offer arbitrary precision give you the ability to be more flexible when you need it. For example, HPF offers the use and control of guard digits down in that basement area. If you need them, they are there to save the digits you need from corruption.
You can use Multiple Precision Toolkit from MATLAB File Exchange for arbitrary precision numbers. Floating point numbers do not usually have a precise base-10 presentation.
That's because your number is beyond the precision of the double numeric type (it gives you between 15 to 17 significant decimal digits). In your case, it is rounded to the nearest representable number as soon as the literal is evaluated.
If you need more precision than what the double-precision floating-points provides, store the numbers in strings, or use arbitrary-precision libraries. For example use the Symbolic Toolbox:
sym('0.0111243254654764549999999')
You cannot get EXACT string since the number is stored in double type, or even long double type.
The number stored will be a subtle more or less than the number you gives.
computer only knows binary number 0 & 1. You must know that numbers in one radix may not expressed the same in other radix. For example, number 1/3, radix 10 yields 0.33333333...(The ellipsis (three dots) indicate that there would still be more digits to come, here is digit 3), and it will be truncated to 0.333333; radix 3 yields 0.10000000, see, no more or less, exactly the amount; radix 2 yields 0.01010101... , so it will likely truncated to 0.01010101 in computer,that's 85/256, less than 1/3 by rounding, and next time you fetch the number, it won't be the same you want.
So from the beginning, you should store the number in string instead of float type, otherwise it will lose precision.
Considering the precision problem, MATLAB provides symbolic computation to arbitrary precision.
I have following JSON response string
{"firstname":"a","lastname":"a","jobtitle":"software developer","companyname":"abc","mobileno":9461438988}
i cant get the string value for 'mobileno' field
This is how i read the 'mobileno' field
self.dispPhone = [NSString stringWithFormat:#"%# ",[parsedProfileData valueForKey:#"mobileno"]];
the string here becomes some garbage value '2147483647' when i NSLog it
First of all, a phone number is not an integer. Phone numbers may start with significant 0s or +. For example, in Germany 00.. is international, 0 national, and any other digit regional.
The returned value is the highest possible 31 bit number. Since your JSON encoder decodes to signed 32bit ints, this is the best value it can give you.
To solve this problem, format phone numbers as strings in the original.
The number 9,461,438,988 cannot be stored in a 32-bit value, so it's being truncated to fit.
You should store your phone numbers as the strings they are (just like ZIP codes).
In general, anything that you don't want to do arithmetic or numeric comparison on (< or >) is a string, not a number.