This question already has answers here:
Is floating point math broken?
(31 answers)
Closed 3 years ago.
I was using a ROUNDUP functions and find a very strange answers and later i found out that 1-1.09 = -0.0900000000000001 according to excel. i am not a mathematician but i think than correct answer 1 - 1.09 must be -0.09. i cant figure it out why the answer is -0.0900000000000001. is this a bug or i have missed something?
P.S. I am using Excel2016 16.0.4417.10000 64-bit, on Windows 10
Computers only have 0 and 1 to represent any value. Converting 0.1 into a binary number would result in a long ‘zeros-and-ones-number’: .0001100011000111000111
In order to avoid an endless number, Excel would round it at the end. But this rounding of the binary equivalent to 0.1 leads to miscalculation.
found it on here. Basically excel does it to save memory https://professor-excel.com/wrong-calculations-why-does-excel-show-a-wrong-result/
Related
This question already has answers here:
Is floating point math broken?
(31 answers)
Closed 2 years ago.
Adding 2 numbers in nodejs with 2 decimal places doesn't leads to unexpected results
The following code results in
console.log(34.02 + 1378.12);
1412.1399999999999
But the expected result is
1412.14
You can use toFixed(2).
console.log((34.02 + 1378.12).toFixed(2));
This question already has answers here:
How to search for a string in cell array in MATLAB?
(8 answers)
Closed 6 years ago.
I'm a newbie to Matlab and have a question about str locating:
A = ['abc','de','fghij','something','another'];
Then how can I get 3 if I use strfind(A,'fghij')?
Thanks.
I'm not exactly sure I understand your question, but if you are wondering why the value is 3 instead of 2 it is because matlab (unlike most languages you may be used to) indexes arrays starting at 1 instead of 0.
Thanks #TroyHaskin ! I find the answer in another post~
idx=find(ismemeber(A,'fghij')) is what I want.
This question already has answers here:
Is floating point math broken?
(31 answers)
Closed 6 years ago.
I've been trying to use Fortran for my research project, with the GNU Fortran compiler (gfortran), latest version,
but I've been encountering some problems in the way it processes real numbers. If you have for example the code:
program test
implicit none
real :: y = 23.234, z
z = y * 100000
write(*,*) y, z
end program
You'll get as output:
23.23999 2323400.0
I find this really strange.
Can someone tell me what's exactly happening here? Looking at z I can see that y does retain its precision, so for calculations that shouldn't be a problem I suppose. But why is the output of y not exactly the same as the value that I've specified, and what can I do to make it exactly the same?
This is not a problem - all you see is floating-point representation of the number in the computer. The computer cannot handle real numbers exactly, but only approximations of them. A good read about this can be found here: What Every Computer Scientist Should Know About Floating-Point Arithmetic.
Simply by replacing real with double precision, you can increase the number of significant decimal places from about six to about 15 on most platforms.
The general issue is not limited to Fortran, but the representation of base 10 real numbers in another base of finite precision. This computer science question is asked many times here.
For the specifically Fortran aspects, the declaration "real" will likely give you a single precision floating point. As will expressing a constant as "23.234" without a type qualifier. The constant "100000" without a decimal point is an integer so the expression "y * 100000" is causing an implicit conversion of an integer to a real because "y" is a real variable.
For previous some previous discussions of these issues see Extended double precision , Fortran: integer*4 vs integer(4) vs integer(kind=4) and Is There a Better Double-Precision Assignment in Fortran 90?
The problem here is not with Fortran, in fact it is not a problem at all. This is just a feature of floating-point arithmetic. If you think about how you would represent 23.234 as a 'single float' in binary, you would see that the number has to be saved to only so many decimals of precision.
The thing to remember about float point number is: numbers that look round and even in base-10 probably won't in binary.
For a brief overview of floating-point topics, check the Wikipedia article. And for a VERY thorough explanation, check out the canonical paper by Goldberg (PDF).
This question already has answers here:
How many objects are created
(4 answers)
Closed 9 years ago.
for this statement
String a="MAM"+"BCD"+"EFG"+"GFE";
How many objects will be created? (I am confused is it created 4 or 5 or 7)
Most smart compilers will realize that concatenated string constants can be concatenated at compile time. If your compiler chooses to make that optimization, the answer is one.
Otherwise, you have each of your literal strings, plus one for each concatenation. Without the optimization, the answer is 7 because you have 4 strings and 3 +es.
If you're talking about Java, the answer is one, at compile time, as specified in the JLS.
If you're not, the question is unanswerable as posed.
Only one object will be created. In this case because the plus sign is used to concatenate "string literals". :)
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Rounding a double to 5 decimal places in Java ME
i have a double value which is 2.8. I was planning to display on screen in the form of 2.80. Is there any idea how to do it?
thanks=)
The DecimalFormat class, in Java SE, can handle this scenario. Unfortunately, Java ME doesn't have it or anything similar to it.
I think the solution is to convert the double value to string and add the extra '0' at the end.
String formattedValue = Double.toString(value).concat("0");