I want my sigmoid to never print a solid 1 or 0, but to actually print the exact value
i tried using
torch.set_printoptions(precision=20)
but it didn't work. here's a sample output of the sigmoid function :
before sigmoid : tensor([[21.2955703735]])
after sigmoid : tensor([[1.]])
but i don't want it to print 1, i want it to print the exact number, how can i force this?
The difference between 1 and the exact value of sigmoid(21.2955703735) is on the order of 5e-10, which is significantly less than machine epsilon for float32 (which is about 1.19e-7). Therefore 1.0 is the best approximation that can be achieved with the default precision. You can cast your tensor to a float64 (AKA double precision) tensor to get a more precise estimate.
torch.set_printoptions(precision=20)
x = torch.tensor([21.2955703735])
result = torch.sigmoid(x.to(dtype=torch.float64))
print(result)
which results in
tensor([0.99999999943577644324], dtype=torch.float64)
Keep in mind that even with 64-bit floating point computation this is only accurate to about 6 digits past the last 9 (and will be even less precise for larger sigmoid inputs). A better way to represent numbers very close to one is to directly compute the difference between 1 and the value. In this case 1 - sigmoid(x) which is equivalent to 1 / (1 + exp(x)) or sigmoid(-x). For example,
x = torch.tensor([21.2955703735])
delta = torch.sigmoid(-x.to(dtype=torch.float64))
print(f'sigmoid({x.item()}) = 1 - {delta.item()}')
results in
sigmoid(21.295570373535156) = 1 - 5.642236648842976e-10
and is a more accurate representation of your desired result (though still not exact).
In some programming competitions where the numbers are larger than any available integer data type, we often use strings instead.
Question 1:
Given these large numbers, how to calculate e and f in the below expression?
(a/b) + (c/d) = e/f
note: GCD(e,f) = 1, i.e. they must be in minimised form. For example {e,f} = {1,2} rather than {2,4}.
Also, all a,b,c,d are large numbers known to us.
Question 2:
Can someone also suggest a way to find GCD of two big numbers (bigger than any available integer type)?
I would suggest using full bytes or words rather than strings.
It is relatively easy to think in base 256 instead of base 10 and a lot more efficient for the processor to not do multiplication and division by 10 all the time. Ideally, choose a word size that is half the processor's natural word size, as that makes carry easy to implement. Of course thinking in base 64K or 4G is slightly more complex, but even better than base 256.
The only downside is generating the initial big numbers from the ascii input, which you get for free in base 10. Using a larger word size you can make this more efficient by processing a number of digits initially into a single word (eg 9 digits at a time into 4G), then performing a long multiply of that single word into the correct offset in your large integer format.
A compromise might be to run your engine in base 1 billion: This will still be 9 or 81 times more efficient than using base 10!
The simplest way to solve this equation is to multiply a/b * d/d and c/d * b/b so they both have the common denominator b*d.
I think you will then need to prime factorise your big numbers e and f to find any common factors. Remember to search again for the same factor squared.
Of course, that means you have to write a prime generating sieve. You only need to generate factors up to the square root, or half the digits of the min value of e and f.
You could prime factorise b and d to get a lower initial denominator, but you will need to do it again anyway after the addition.
I think that the way to solve this is to separate the problem:
Process the input numbers as an array of characters (ie. std::string)
Make a class where each object can store an std::list (or similar) that represents one of the large numbers, and can do the needed arithmetic with your data
You can then solve your problems normally, without having to worry about your large inputs causing overflow.
Here's a webpage that explains how you can have such an arithmetic class (with sample code in C++ showing addition).
Once you have such an arithmetic class, you no longer need to worry about how to store the data or any overflow.
I get the impression that you already know how to find the GCD when you don't have overflow issues, but just in case, here's an explanation of finding the GCD (with C++ sample code).
As for the specific math problem:
// given formula: a/b + c/d = e/f
// = ( ( a*d + b*c ) / ( b*d ) )
// Define some variables here to save on copying
// (I assume that your class that holds the
// large numbers is called "ARITHMETIC")
ARITHMETIC numerator = a*d + b*c;
ARITHMETIC denominator = b*d;
ARITHMETIC gcd = GCD( numerator , denominator );
// because we know that GCD(e,f) is 1, this implies:
ARITHMETIC e = numerator / gcd;
ARITHMETIC f = denominator / gcd;
I have a large spreadsheet with a number of forumlas and they all make complete sense apart from one, which is listed below. Does anyone have any idea what this NORMALDIST calculation is trying to acheive or tell me? It has relevants to HE
=MAX(1,NORMDIST(3,N18,N18/4,TRUE)-NORMDIST(0,N18,N18/4,TRUE) + 2*(NORMDIST(6,N18,N18/4,TRUE)-NORMDIST(3,N18,N18/4,TRUE)) + 3*(NORMDIST(9,N18,N18/4,TRUE)-NORMDIST(6,N18,N18/4,TRUE)) + 4*(NORMDIST(12,N18,N18/4,TRUE)-NORMDIST(9,N18,N18/4,TRUE)) + 5*(NORMDIST(15,N18,N18/4,TRUE)-NORMDIST(12,N18,N18/4,TRUE)) + 6*(NORMDIST(18,N18,N18/4,TRUE)-NORMDIST(15,N18,N18/4,TRUE)) + 7*(NORMDIST(21,N18,N18/4,TRUE)-NORMDIST(18,N18,N18/4,TRUE)) + 8*(NORMDIST(24,N18,N18/4,TRUE)-NORMDIST(21,N18,N18/4,TRUE)) + 9*(NORMDIST(27,N18,N18/4,TRUE)-NORMDIST(24,N18,N18/4,TRUE)) + 10*(NORMDIST(30,N18,N18/4,TRUE)-NORMDIST(27,N18,N18/4,TRUE)) + 11*(NORMDIST(33,N18,N18/4,TRUE)-NORMDIST(30,N18,N18/4,TRUE)) + 12*(NORMDIST(36,N18,N18/4,TRUE)-NORMDIST(33,N18,N18/4,TRUE)) + 13*(NORMDIST(39,N18,N18/4,TRUE)-NORMDIST(36,N18,N18/4,TRUE)) + 14*(NORMDIST(42,N18,N18/4,TRUE)-NORMDIST(39,N18,N18/4,TRUE)) + 15*(NORMDIST(45,N18,N18/4,TRUE)-NORMDIST(42,N18,N18/4,TRUE)) + 16*(NORMDIST(48,N18,N18/4,TRUE)-NORMDIST(45,N18,N18/4,TRUE)) + 17*(NORMDIST(51,N18,N18/4,TRUE)-NORMDIST(48,N18,N18/4,TRUE)) + 18*(NORMDIST(54,N18,N18/4,TRUE)-NORMDIST(51,N18,N18/4,TRUE)) + 19*(NORMDIST(57,N18,N18/4,TRUE)-NORMDIST(54,N18,N18/4,TRUE)) + 20*(NORMDIST(60,N18,N18/4,TRUE)-NORMDIST(57,N18,N18/4,TRUE)) + 21*(NORMDIST(63,N18,N18/4,TRUE)-NORMDIST(60,N18,N18/4,TRUE)) + 22*(NORMDIST(66,N18,N18/4,TRUE)-NORMDIST(63,N18,N18/4,TRUE)) + 23*(NORMDIST(69,N18,N18/4,TRUE)-NORMDIST(66,N18,N18/4,TRUE)))
Strange question I know, but could not think of where else to ask!!!
Cheers
The equation has a series of terms of the form N*[NORMDIST(3N,mu,sigma)-NORMDIST(3N-3,mu,sigma)] where mu is the mean (N18 in the equation), sigma is the standard deviation (N18/4), and with N going from 1 to 23. This appears to be an estimate involving the average of the normal distribution. It would be more rigorous for N to go from minus infinity to plus infinity and it's not clear why this formula truncated the interval to 1..23. Nevertheless, if the person who wrote the equation was calculating the average, then from the properties of the normal distribution you can derive a closed form solution as:
Total of all NORMDIST terms = mu/3 + 1/2
This will be accurate as long as mu (N18) is in the between 0 and 30. If you plug this into the equation you get
=MAX(1,N18/3+0.5)
Hope that helps.
From the docs...
NORMDIST function
Excel for Office 365 Excel for Office 365 for Mac Excel 2019 Excel 2016 More...
Returns the normal distribution for the specified mean and standard deviation. This function has a very wide range of applications in statistics, including hypothesis testing.
Important: This function has been replaced with one or more new functions that may provide improved accuracy and whose names better reflect their usage. Although this function is still available for backward compatibility, you should consider using the new functions from now on, because this function may not be available in future versions of Excel.
For more information about the new function, see NORM.DIST function.
Does anyone know why the pmulhrsw instruction or
_mm_mulhrs_epi16(x) := RoundDown((x * y + 16384) / 32768)
always rounds towards positive infinity? To me, this is terribly biased for negative numbers, because then a sequence like -0.6, 0.6, -0.6, 0.6, ... won't add up to 0 on average.
Is this behavior intentional or unintentional? If it's intentional, what could be the use? Is there an easy way to make it less biased?
Lucky for me, I can just change the order of my operations to get a less biased result (my function is a signed geometric mean):
__m128i ChooseSign(x, sign)
{
return _mm_sign_epi16(x, sign)
}
signsDifferent = _mm_srai_epi16(_mm_xor_si128(a, b), 15) // (a ^ b) >> 15
sign = _mm_andnot_si128(signsDifferent, a) // !signsDifferent & a
//result = ChooseSign(sqrt(a * b), sign) * fraction // biased
result = ChooseSign(sqrt(a * b) * fraction, sign)
A most serious mistake. I asked the same question on the Intel developer forums and andysem corrected me, pointing out the behavior is to round to the nearest integer.
I was mistaken into thinking it was biased because the formula from MSDN, https://msdn.microsoft.com/en-us/library/bb513995.aspx
was (x * y + 16384) >> 15. This looked very similar to the int(x + 0.5) method for rounding, which I know is biased for negative #s and cringe at. But I didn't realize right shift for negative numbers isn't the same as dividing and casting to an int.
Plus, it wasn't matching my non-SIMD reference implementation, which turns out to be biased since I was calculating int(sum / 9.0f), which rounds towards zero.
I should've had more doubt before questioning the behavior of something implemented in hardware, which would've been rigorously vetted, since int(x + 0.5) would be a very expensive mistake.
_mm_mulhrs_epi16() still has some bias of always rounding x.5 towards + infinity. But that's not a big deal for my application.
I have a class implementing an audio stream that can be read at varying speed (including reverse and fast varying / "scratching")... I use linear interpolation for the read part and everything works quite decently..
But now I want to implement writing to the stream at varying speed as well and that requires me to implement a kind of "reverse interpolation" i.e. Deduce the input sample vector Z that, interpolated with vector Y will produce the output X (which I'm trying to write)..
I've managed to do it for constant speeds, but generalising for varying speeds (e.g accelerating or decelerating) is proving more complicated..
I imagine this problem has been solved repeatedly, but I can't seem to find many clues online, so my specific question is if anyone has heard of this problem and can point me in the right direction (or, even better, show me a solution :)
Thanks!
I would not call it "reverse interpolation" as that does not exists (my first thought was you were talking about extrapolation!). What you are doing is still simply interpolation, just at an uneven rate.
Interpolation: finding a value between known values
Extrapolation: finding a value beyond known values
Interpolating to/from constant rates is indeed much much simpler than the generic quest of "finding a value between known values". I propose 2 solutions.
1) Interpolate to a significantly higher rate, and then just sub-sample to the nearest one (try adding dithering)
2) Solve the generic problem: for each point you need to use the neighboring N points and fit a order N-1 polynomial to them.
N=2 would be linear and would add overtones (C0 continuity)
N=3 could leave you with step changes at the halfway point between your source samples (perhaps worse overtones than N=2!)
N=4 will get you C1 continuity (slope will match as you change to the next sample), surely enough for your application.
Let me explain that last one.
For each output sample use the 2 previous and 2 following input samples. Call them S0 to S3 on a unit time scale (multiply by your sample period later), and you are interpolating from time 0 to 1. Y is your output and Y' is the slope.
Y will be calculated from this polynomial and its differential (slope)
Y(t) = At^3 + Bt^2 + Ct + D
Y'(t) = 3At^2 + 2Bt + C
The constraints (the values and slope at the endpoints on either side)
Y(0) = S1
Y'(0) = (S2-S0)/2
Y(1) = S2
Y'(1) = (S3-S1)/2
Expanding the polynomial
Y(0) = D
Y'(0) = C
Y(1) = A+B+C+D
Y'(1) = 3A+2B+C
Plugging in the Samples
D = S1
C = (S2-S0)/2
A + B = S2 - C - D
3A+2B = (S3-S1)/2 - C
The last 2 are a system of equations that are easily solvable. Subtract 2x the first from the second.
3A+2B - 2(A+B)= (S3-S1)/2 - C - 2(S2 - C - D)
A = (S3-S1)/2 + C - 2(S2 - D)
Then B is
B = S2 - A - C - D
Once you have A, B, C and D you can put in an time 't' in the polynomial to find a sample value between your known samples.
Repeat for every output sample, reuse A,B,C&D if the next output sample is still between the same 2 input samples. Calculating t each time is similar to Bresenham's line algorithm, you're just advancing by a different amount each time.