I have an app that's been in the store since iOS 9. Recently it just broke because of rounding errors. I'm using Xcode Version 9.2 (9C40b). I wrote a piece of text code that shows rounding working, in iOS 9, but not iOS 11. The result should be rounded to 2 decimal places.
NSDecimalNumberHandler* round = [NSDecimalNumberHandler decimalNumberHandlerWithRoundingMode:NSRoundPlain scale:2 raiseOnExactness:NO raiseOnOverflow:NO raiseOnUnderflow:NO raiseOnDivideByZero:YES];
NSDecimalNumber* myNumber = [NSDecimalNumber decimalNumberWithString:#"67.95999999999997"];
NSDecimalNumber* result = [myNumber decimalNumberByRoundingAccordingToBehavior:round];
NSLog(#"myNumber = %#", myNumber);
NSLog(#"rounding result = %#", result);
On iOS 11:
myNumber = 67.95999999999997
rounding result = 67.96000000000001
**On iOS9 **
myNumber = 67.95999999999997
rounding result = 67.96
UPDATE - Problem Solved
This isn't a rounding error, but an issue with NSDecimalNumber's description method. When I looked in the value, with the debugger, it's correct.
To see the correct result use:
NSLog(#"rounding result = %#", result.stringValue);
Related
I am saving values to a .csv file in NXC(Not eXactly C) and then calling on them ata later point in time. The problem I am having is when calling any negative values back from a cell it is displayed as 0123 instead of -123 which is throwing all my additional calculations off.
The current code is:
OpenFileRead("map.csv", fSize, count);
until (eof == true) {
ReadLnString(count, val);
int lstFwd = StrToNum(val);
NumOut(0,LCD_LINE1,lstFwd);
}
while(true);
Can anyone explain how to rectify this issue as it is causing me a great deal of stress now.
StrToNum should convert negativ numbers. Its a bit strange that an integer number starts with 0. You should also use Enhanced NBC/NXC firmware.
First: You should always clear the screen before writing some output!
Use:
NumOut(0,LCD_LINE1,lstFwd, DRAW_OPT_CLEAR_LINE);
If the problem still exists try:
string val;
OpenFileRead("map.csv", fSize, count);
until (eof == true) {
ReadLnString(count, val);
int lstFwd = StrToNum(val);
if(SubStr(val, 0, 1) == "-") lstFwd *= -1; // Check if first char is "-"
NumOut(0,LCD_LINE1,lstFwd, DRAW_OPT_CLEAR_LINE);
}
while(true);
In the following question/answer:
Configure Cart Price Update
The response gives an example that provides a working method - however, I had to change it to allow for floating point math.
The problem now is that I cannot convert the answer to 2 decimal places. The answer is sometimes given with 15! decimal places.
Here is my modified java code from the original - how do I get it to output to 2 decimals?
<script type="text/javascript">
$(document).ready(function(){
var basePrice = 39.95;
$("#baseCost").text(basePrice);
$("#sumTotal").text(basePrice);
});
function calculateTotals(){
var basePrice = parseFloat($("#baseCost").text(), 10);
var upgradePrice = 0.00;
$("#options select").each(function(){
var optionVal = $(this).val();
upgradePrice += parseFloat(optionVal.substr(optionVal.indexOf("_") + 1, optionVal.length - 1), 10);
});
$("#upgradeCost").text(upgradePrice);
$("#sumTotal").text(basePrice + upgradePrice);
}
</script>
Every number variable has a method called toFixed.
var sumTotal = basePrice + upgradePrice;
$("#sumTotal").text(sumTotal.toFixed(2));
Edit: Changed answer to something more specific.
I am experiencing some problems with a line of C# code.
I'm trying to get the end customer amount of a price in our database. It is a nullable decimal so if it isn't filled in, I use 0.
Could someone look at this line of code and give me an explanation to why this doesn't work?
This is the specific line:
Decimal totalPrice = requestedPrice.EndCustomerAmount.HasValue ? requestedPrice.EndCustomerAmount.Value : 0;
The problem is that for some prices, the totalPrice is 0 even when the EndCustomerAmount has a value.
If I debug the code and I execute the if statement in the immediate window, it returns the correct value. Even when I assign the value in the immediate window the totalPrice variable holds the correct amount.
I have tried following lines to solve the problem but with no luck:
Decimal totalPrice = requestedPrice.EndCustomerAmount ?? 0;
And this:
Decimal totalPrice = requestedPrice.EndCustomerAmount ?? 0m;
And this:
Decimal totalPrice = 0
totalPrice = requestedPrice.EndCustomerAmount.HasValue ? requestedPrice.EndCustomerAmount.Value : 0;
What does seem to work is this:
Decimal totalPrice = 0
if(requestedPrice.EndCustomerAmount.HasValue)
totalPrice = requestedPrice.EndCustomerAmount
Or this:
Decimal? totalPrice = requestedPrice.EndCustomerAmount.HasValue ? requestedPrice.EndCustomerAmount.Value : 0;
Thanks!
I found a blog post explaining the debugger issues, including a response from Microsoft:
http://geekswithblogs.net/twickers/archive/2011/03/31/misreporting-of-variable-values-when-debugging-x64-code-with-the.aspx
Tl;dr: It's an issue with the ?? and ?: operators on nullable struct types larger than 64 bits (Decimal, Guid, ...) in the 64 bit CLR JIT where their value is not updated until the next statement. It has been fixed in VS2012.
If you're interested in a simplistic reproduction, just run this in 64x debug in VS2010:
Decimal? foo = 10.5m;
var result = foo.HasValue ? foo.Value : 0;
Console.WriteLine(result);
Put your breakpoint on the Console.Writeline line. Hover over result and see the 0. If we defined and assigned a value to result earlier, it would still show the old value. When using result in watches of immediate window execution it will still use the old value. Moving to the next line, you'll see the value updates.
I got a wrong result in an equation that I used in my following code
dQ_rad = 0.7 * 5.67e-8 * rotor.dRotorOuterArea[iAxle] * (dT1*dT1*dT1*dT1 - dT2*dT2*dT2*dT2);
All the variables are declared as DOUBLE, where
rotor.dRotorOuterArea[iAxle] = 0.052986887100527499
dT1 = 0;
dT2 = 293.0;
dQ_rad will get the result -1.#IND000000000000, which I really don't understand.
Then I used the same equation in "QuickWatch", the correct result can be seen as follows (!?)
0.7 * 5.67e-8 * rotor.dRotorOuterArea[iAxle] * (dT1*dT1*dT1*dT1 - dT2*dT2*dT2*dT2) -15.499582013297069 double
Does anyone know how this error happened and how I can avoid this kind of error?
I use VC Express 2010 and the code is compiled using default MS C-Compiler.
Many thanks
-1.#IND000000000000 is Microsoft's representation of NaN. NaN can result due to a variety of operations, such as sqrt(-1.0), log(-1.0), 0/0, 0*INF, INF/INF. NaN is also propagated, so any operation on a double with value NaN will resut in a NaN. The following article provides more information about this (and other floating point states), and may be valuable in debugging this problem:
http://www.johndcook.com/IEEE_exceptions_in_cpp.html
Unfortunately, I wasn't able to reproduce your error in a trivial test case so it is difficult to provide more detailed information:
#include <stdio.h>
int main(void) {
double rotor = 0.052986887100527499;
double dT1 = 0;
double dT2 = 293.0;
double dQ_rad = 0.7 * 5.67e-8 * rotor * (dT1*dT1*dT1*dT1 - dT2*dT2*dT2*dT2);
//fprintf(stderr, "%.12lf\n", dQ_rad);
return 0;
}
One option you could try is to split your dQ_rad calculation into multiple steps, and verify the result of each operation to make sure it is not NaN. Something like:
double dQ_rad1 = 0.7 * 5.67e-18;
double dQ_rad2 = dQ_rad1 * rotor;
double dQ_rad3 = dT1*dT1*dT1*dT1;
double dQ_rad4 = dT2*dT2*dT2*dT2;
double dQ_rad5 = dQ_rad3 - dQ_rad4;
double dQ_rad = dQ_rad2 * dQ_rad5;
This may help to isolate which part of the calculation is resulting in a NaN.
I am using Actionscript 2.0
In a Brand new Scene. My only bit of code is:
trace(int('04755'));
trace(int('04812'));
Results in:
2541
4812
Any idea what I am doing wrong/silly?
By the way, I am getting this source number from XML, where it already has the leading 0. Also, this works perfect in Actionscript 3.
In AS3, you can try:
parseInt('04755', 10)
10 above is the radix.
parseInt(yourString);
...is the correct answer. .parseInt() is a top-level function.
Converting a string with a leading 0 to a Number in ActionScript 2 assumes that the number you want is octal. Give this function I've made for you a try:
var val:String = '00010';
function parse(str:String):Number
{
for(var i = 0; i < str.length; i++)
{
var c:String = str.charAt(i);
if(c != "0") break;
}
return Number(str.substr(i));
}
trace(parse(val)); // 10
trace(parse(val) + 10); // 20
Basically what you want to do now is just wrap your string in the above parse() function, instead of int() or Number() as you would typically.
Bit of a simple one...
try this -
temp="120";
temp2="140";
temp3=int ( temp );
temp4=int ( temp2 );
temp5=temp4+temp3;
trace(temp5);
so, all you need is...
int("190");