Different values depending on floating point exception flags set - visual-c++

Question

Short question:
How can setting the _EM_INVALID exception flag on the FPU result in different values?
Long question:
In our project we have turned off floating point exceptions in our Release build, but turned on ZERODIVIDE, INVALID and OVERFLOW using _controlfp_s() in our Debug build. This is in order to catch errors if they are there.
However, we would also like results of numerical calculations (involving optimisation algorithms, matrix inversion, Monte Carlo and all sorts of things) to be consistent between Debug and Release build to make debugging easier.
I would expect that the setting of the exception flags on the FPU should not affect the calculated values - only whether exceptions are thrown or not. But after working backwards through our calculations I can isolate the below code example that shows that there is a difference on the last bit when calling the log() function.
This propagates to a 0.5% difference in the resulting value.
The below code will give the shown program output when adding it to a new solution in Visual Studio 2005, Windows XP and compile in Debug configuration. (Release will give a different output, but that's because the optimiser reuses the result from the first call to log().)
I hope that someone can shed a bit of light on this. Thanks.
/*
Program output:
Xi, 3893f76f, 7.4555176582633598
K, c0a682c7, 7.44466687218
Untouched
x, da8caea1, 0.0014564635732296288
Invalid exception on
x, da8caea2, 0.001456463573229629
Invalid exception off
x, da8caea1, 0.0014564635732296288
*/
#include <float.h>
#include <math.h>
#include <limits>
#include <iostream>
#include <iomanip>
using namespace std;
int main()
{
unsigned uMaskOld = 0;
errno_t err;
cout << std::setprecision (numeric_limits<double>::digits10 + 2);
double Xi = 7.4555176582633598;
double K = 7.44466687218;
double x;
cout << "Xi, " << hex << setw(8) << setfill('0') << *(unsigned*)(&Xi) << ", " << dec << Xi << endl;
cout << "K, " << hex << setw(8) << setfill('0') << *(unsigned*)(&K) << ", " << dec << K << endl;
cout << endl;
cout << "Untouched" << endl;
x = log(Xi/K);
cout << "x, " << hex << setw(8) << setfill('0') << *(unsigned*)(&x) << ", " << dec << x << endl;
cout << endl;
cout << "Invalid exception on" << endl;
::_clearfp();
err = ::_controlfp_s(&uMaskOld, 0, _EM_INVALID);
x = log(Xi/K);
cout << "x, " << hex << setw(8) << setfill('0') << *(unsigned*)(&x) << ", " << dec << x << endl;
cout << endl;
cout << "Invalid exception off" << endl;
::_clearfp();
err = ::_controlfp_s(&uMaskOld, _EM_INVALID, _EM_INVALID);
x = log(Xi/K);
cout << "x, " << hex << setw(8) << setfill('0') << *(unsigned*)(&x) << ", " << dec << x << endl;
cout << endl;
return 0;
}

Answer 1

This is not a complete answer, but it is too long for a comment.
I suggest you isolate the code that does the questionable calculations and put it in a subroutine, preferably in a source module that is compiled separately. Something like:
void foo(void)
{
double Xi = 7.4555176582633598;
double K = 7.44466687218;
double x;
x = log(Xi/K);
…Insert output statements here…
}
Then you would call the routine with different settings:
cout << "Untouched:\n";
foo();
cout << "Invalid exception on:\n";
…Change FP state…
foo();
This guarantees that the same instructions are executed in each case, eliminating the possibility that the compiler has for some reason generated separate code for each sequence. The way you have compiled the code, I suspect it is possible the compiler may have used 80-bit arithmetic in one case and 64-bit arithmetic in another, or may have used 80-bit arithmetic generally but converted some result to 64-bit in one case but not another
Once that is done, you can partition and isolate the code further. E.g., try evaluating Xi/K once before any of the tests, storing that in a double, and passing it to foo as a parameter. The tests whether the log call differs depending on the floating-point state. I suspect that is the case, as it is unlikely the division operation would differ.
Another advantage of isolating the code this way is that you could step through it in the debugger to see exactly where behavior diverges. You could step through it, one instruction at a time, with different floating-point states simultaneously in two windows and examine the results at each step to see exactly where the divergence is. If there is no divergence by the time you reach the log call, you should step through that, too.
Incidental notes:
If you know Xi and K are close to each other, it is better to compute log(Xi/K) as log1p((Xi-K)/K). When Xi and K are close to each other, the subtraction Xi-K is exact (has no error), and the quotient has more useful bits (the 1 that we already knew about and some zero bits following it are gone).
The fact that slight changes in your floating-point environment cause a .5% change in your result implies your calculations are very sensitive to error. This suggests that, even if you make your results reproducible, the errors that necessarily exist in floating-point arithmetic cause your result to be inaccurate. That is, the final error will still exist, it just will not be called to your attention by the difference between two different ways of calculating.
It appears in your C++ implementation that unsigned is four bytes but double is eight bytes. So printing the encoding a double by aliasing it to an unsigned omits half of the bits. Instead, you should convert a pointer to the double to a pointer to const char and print sizeof(double) bytes.