numpy's round int doesn't seem to be consistent with how it deals with xxx.5
In [2]: np.rint(1.5)
Out[2]: 2.0
In [3]: np.rint(10.5)
Out[3]: 10.0
1.5 is rounded up while 10.5 is rounded down. Is there a reason for this? Is it just and artifact of the inaccuracy of floats?
Edit
Is there a way to get the desired functionality where n.5 is rounded up i.e. to n+1 for both n = even or odd?
So, this kind of behavior (as noted in comments), is a very traditional form of rounding, seen in the round half to even method. Also known (according to David Heffernan) as banker's rounding. The numpy documentation around this behavior implies that they are using this type of rounding, but also implies that there may be issues with the way in which numpy interacts with the IEEE floating point format. (shown below)
Notes
-----
For values exactly halfway between rounded decimal values, Numpy
rounds to the nearest even value. Thus 1.5 and 2.5 round to 2.0,
-0.5 and 0.5 round to 0.0, etc. Results may also be surprising due
to the inexact representation of decimal fractions in the IEEE
floating point standard [1]_ and errors introduced when scaling
by powers of ten.
Whether or not that is the case, I honestly don't know. I do know that large portions of the numpy core are still written in FORTRAN 77, which predates the IEEE standard (set in 1984), but I don't know enough FORTRAN 77 to say whether or not there's some issue with the interface here.
If you're looking to just round up regardless, the np.ceil function (ceiling function in general), will do this. If you're looking for the opposite (always rounding down), the np.floor function will achieve this.
This is in fact exactly the rounding specified by the IEEE floating point standard IEEE 754 (1985 and 2008). It is intended to make rounding unbiased. In normal probability theory, a random number between two integers has zero probability of being exactly N + 0.5, so it shouldn't matter how you round it because that case never happens. But in real programs, numbers are not random and N + 0.5 occurs quite often. (In fact, you have to round 0.5 every time a floating point number loses 1 bit of precision!) If you always round 0.5 up to the next largest number, then the average of a bunch rounded numbers is likely to be slightly larger than the average of the unrounded numbers: this bias or drift can have very bad effects on some numerical algorithms and make them inaccurate.
The reason rounding to even is better than rounding to odd is that the last digit is guaranteed to be zero, so if you have to divide by 2 and round again, you don't lose any information at all.
In summary, this kind of rounding is the best that mathematicians have been able to devise, and you should WANT it under most circumstances. Now all we need to do is get schools to start teaching it to children.
Numpy rounding does round towards even, but the other rounding modes can be expressed using a combination of operations.
>>> a=np.arange(-4,5)*0.5
>>> a
array([-2. , -1.5, -1. , -0.5, 0. , 0.5, 1. , 1.5, 2. ])
>>> np.floor(a) # Towards -inf
array([-2., -2., -1., -1., 0., 0., 1., 1., 2.])
>>> np.ceil(a) # Towards +inf
array([-2., -1., -1., -0., 0., 1., 1., 2., 2.])
>>> np.trunc(a) # Towards 0
array([-2., -1., -1., -0., 0., 0., 1., 1., 2.])
>>> a+np.copysign(0.5,a) # Shift away from 0
array([-2.5, -2. , -1.5, -1. , 0.5, 1. , 1.5, 2. , 2.5])
>>> np.trunc(a+np.copysign(0.5,a)) # 0.5 towards higher magnitude round
array([-2., -2., -1., -1., 0., 1., 1., 2., 2.])
In general, numbers of the form n.5 can be accurately represented by binary floating point (they are m.1 in binary, as 0.5=2**-1), but calculations expected to reach them might not. For instance, negative powers of ten are not exactly represented:
>>> (0.1).as_integer_ratio()
(3602879701896397, 36028797018963968)
>>> [10**n * 10**-n for n in range(20)]
[1, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0,
0.9999999999999999, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
Numpy uses bankers rounding so .5 is rounded to the nearest even number. If you always want to round .5 up but round .4 down:
np.rint(np.nextafter(a, a+1))
or if you always want to round .5 down and .4 down but .6 up:
np.rint(np.nextafter(a, a-1))
NB this also works with np.around if you want the same logic but not integers.
>>> a = np.array([1, 1.5, 2, 2.5, 3, 3.5])
>>> np.rint(a)
array([1., 2., 2., 2., 3., 4.])
>>> np.rint(np.nextafter(a, a+1))
array([1., 2., 2., 3., 3., 4.])
>>> np.rint(np.nextafter(a, a-1))
array([1., 1., 2., 2., 3., 3.])
What is happening?
nextafter gives the next representable number in a direction, so this is enough to push the number off 'exactly' 2.5.
Note this is different to ceil and floor.
>>> np.ceil(a)
array([1., 2., 2., 3., 3., 4.])
>>> np.floor(a)
array([1., 1., 2., 2., 3., 3.])
An answer to you edit:
y = int(np.floor(n + 0.5))
The built-in round function seems to do what you want, although it only works on scalars:
def correct_round(x):
try:
y = [ round(z) for z in x ]
except:
y = round(x)
return y
and then to verify:
print correct_round([-2.5,-1.5,-0.5,0.5,1.5,2.5])
> [-3.0, -2.0, -1.0, 1.0, 2.0, 3.0]
Not sure its the most efficient solution but it works:
signs = np.sign(arr)
tmp = signs * arr
arr = np.floor(tmp + 0.5)
arr = arr * signs
round half up function for for scalar, list and numpy array:
import numpy as np
def round_half_up(x):
round_lambda = lambda z: (int(z > 0) - int(z < 0)) * int(abs(z) + 0.5)
if isinstance(x, (np.ndarray, np.generic)):
return np.vectorize(round_lambda)(x)
else:
return round_lambda(x)
This worked for me:
def my_round(a):
return np.round(a)*(a-np.floor(a)!=0.5) + np.ceil(a)*(a-np.floor(a)==0.5)
>>> my_round([0.5, 1.5, 2.5, 3.5])
array([1., 2., 3., 4.])
Related
I'm looking to transition from Matlab to Python with numpy. I am having some difficulties understanding array indexing using numpy.
As an example, I'm looking to use a 'for' loop to assign the inner elements of a row vector to be a specific value. Indexing in numpy seems to be unique to when it comes to indexing rank 1 column arrays versus row arrays.
# Python:
import nump as np
from numpy import*
import scipy.linalg
N = 5
L = 1.0
dx = L / N
S = 10.0 ** -2.0
k = 500.0
aW = zeros((1, N))
for i in range(1, (N-1)):
aE[0, i] = k * S / dx
% Matlab:
N=5;
L=1.0;
dx=L/N;
S=10^(-2.0);
k=500.0;
aW=zeros(1,N);
for i = 2:N-1
aE(i)=k*S/dx;
end
Is it necessary for me to specify the row index of 0, instead of just stating:
'aE[i] = k * S / dx'
Matlab doesn't seem to care about the dimensions of the matrix when assigning elements based on an single input for the index.
I don't have an issue with stating the row index. It makes me more conscious of my variable dimensions. I just want verification that it's necessary. Perhaps I'm setting up the vector incorrectly. I'd appreciate the help.
You should try to avoid iterating using for loops, instead try to solve the problem using broadcasting.
Read about numpy broadcasting
https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html
You can do this for example.
>>> import numpy as np
>>> np.full((2,3), 10)
array([[10, 10, 10],
[10, 10, 10]])
or
>>> a = np.ones((2,3))
>>> a
array([[1., 1., 1.],
[1., 1., 1.]])
>>> a * 10
array([[10., 10., 10.],
[10., 10., 10.]])
or
>>> b = np.zeros((2,3))
>>> b
array([[0., 0., 0.],
[0., 0., 0.]])
>>> b+10
array([[10., 10., 10.],
[10., 10., 10.]])
I am getting unexpected non-sparse results when using the kron method of Scipy's sparse module. Specifically, matrix elements that are equal to zero after performing the kronecker product are being kept in the result, and I'd like to understand what I should do to ensure the output is still fully sparse.
Here's an example of what I mean, taking the kronecker product of two copies of the identity:
import scipy.sparse as sp
s = sp.eye(2)
S = sp.kron(s,s)
S
<4x4 sparse matrix of type '<class 'numpy.float64'>'
with 8 stored elements (blocksize = 2x2) in Block Sparse Row format>
print(S)
(0, 0) 1.0
(0, 1) 0.0
(1, 0) 0.0
(1, 1) 1.0
(2, 2) 1.0
(2, 3) 0.0
(3, 2) 0.0
(3, 3) 1.0
The sparse matrix S should only contain the 4 (diagonal) non-zero entries, but here it also has other entries that are equal to zero. Any pointers on what I am doing wrong would be much appreciated.
In
Converting from sparse to dense to sparse again decreases density after constructing sparse matrix
I point out that sparse.kron produces, by default a BSR format matrix. That's what your display shows. Those extra zeros are part of the dense blocks.
If you specify another format, kron will not produce those zeros:
In [672]: sparse.kron(s,s,format='csr')
Out[672]:
<4x4 sparse matrix of type '<class 'numpy.float64'>'
with 4 stored elements in Compressed Sparse Row format>
In [673]: _.A
Out[673]:
array([[1., 0., 0., 0.],
[0., 1., 0., 0.],
[0., 0., 1., 0.],
[0., 0., 0., 1.]])
Below code :
ux = torch.tensor(np.array([[255,1,255],[255,1,255]])).float()
print(ux)
ux = F.normalize(ux, p=2, dim=1)
print(ux)
prints :
tensor([[ 255., 1., 255.],
[ 255., 1., 255.]])
tensor([[ 0.7071, 0.0028, 0.7071],
[ 0.7071, 0.0028, 0.7071]])
How can I un-normalize the ux in order to return to values
tensor([[ 255., 1., 255.],
[ 255., 1., 255.]])
from
tensor([[ 0.7071, 0.0028, 0.7071],
[ 0.7071, 0.0028, 0.7071]])
There are various resources that detail this process such as https://discuss.pytorch.org/t/simple-way-to-inverse-normalize-a-batch-of-input-variable/12385/3 but do not detail unnormalizing result of F.normalize
F.normalize simply divides by the norm according to the documentation, so you simply need to multiply it by its magnitude.
This means you still need access to the magnitude of the original vector ux, otherwise, this is not possible, since the information about the magnitude cannot be recovered from the normalized vector.
Here's how this can be done:
# I modified the input to make it more interesting, but you can use any other value
ux = torch.tensor(np.array([[255,1,255],[101,10,123]])).float()
magnitude = ux.norm(p=2, dim=1, keepdim=True) # NEW
ux = F.normalize(ux, p=2, dim=1)
ux_orig = ux * magnitude # NEW
print(ux_orig)
# Outputs:
# tensor([[255., 1., 255.],
# [101., 10., 123.]])
Can someone explain why the OneVsRestClassifier gives different result than the out-of-the-box algorithm?
from sklearn.multiclass import OneVsRestClassifier, OneVsOneClassifier
X = [[1,2],[1,3],[4,2],[2,3],[1,4]]
y = [1,2,3,2,1]
X_pred = [[2,4], [5,4], [3,7]]
dummy_clf = OneVsRestClassifier(SGDClassifier(verbose=0, class_weight="auto", loss='modified_huber', random_state=0)) # first case
#dummy_clf = SGDClassifier(verbose=0, class_weight="auto", loss='modified_huber', random_state=0) # second case
dummy_clf.fit(X, y)
dummy_clf.predict_proba(X_pred)
First case:
array([[ 0.5, 0.5, 0. ],
[ 0. , 1. , 0. ],
[ 0.5, 0.5, 0. ]])
Second case:
array([[ 0., 1., 0.],
[ 0., 1., 0.],
[ 0., 1., 0.]])
OneVsRest gives you the probability of X_pred for all of the classes, thus the first and last test cases have a value for multiple classes (that sum to 1). The classifier is trained on all classes.
OneVsOne trains a classifier on all class pairs. For all class pairs, the class predicted most is the winner, so you only get one prediction per instance.
Context: I would like to use numpy ndarrays with float32 instead of float64.
Edit: Additional context - I'm concerned about how numpy is executing these calls because they will be happening repeatedly as part of a backpropagation routine in a neural net. I'd like the net to carry out all addition/subtraction/multiplication/division in float32 for validation purposes, as I want to compare results with another group's work. It seems like initialization for methods like randn will always go from float64 -> float32 with .astype() casting. Once my ndarray is of type float32 if i use np.dot for example will those multiplications happen in float32? How can I verify?
The documentation is not clear to me - http://docs.scipy.org/doc/numpy/reference/generated/numpy.dot.html
I figured out I can just add .astype('float32') to the end of a numpy call, for example, np.random.randn(y, 1).astype('float32').
I also see that dtype=np.float32 is an option, for example, np.zeros(5, dtype=np.float32). However, trying np.random.randn((y, 1), dtype=np.float32) returns the following error:
b = np.random.randn((3,1), dtype=np.float32)
TypeError: randn() got an unexpected keyword argument 'dtype'
What is the difference between declaring the type as float32 using dtype and using .astype()?
Both b = np.zeros(5, dtype=np.float32) and b = np.zeros(5).astype('float32') when evaluated with:
print(type(b))
print(b[0])
print(type(b[0]))
prints:
[ 0. 0. 0. 0. 0.]
<class 'numpy.ndarray'>
0.0
<class 'numpy.float32'>
Let's see if I can address some of the confusion I'm seeing in the comments.
Make an array:
In [609]: x=np.arange(5)
In [610]: x
Out[610]: array([0, 1, 2, 3, 4])
In [611]: x.dtype
Out[611]: dtype('int32')
The default for arange is to make an int32.
astype is an array method; it can used on any array:
In [612]: x.astype(np.float32)
Out[612]: array([ 0., 1., 2., 3., 4.], dtype=float32)
arange also takes a dtype parameter
In [614]: np.arange(5, dtype=np.float32)
Out[614]: array([ 0., 1., 2., 3., 4.], dtype=float32)
whether it created the int array first and converted it, or made the float32 directly isn't any concern to me. This is a basic operation, done in compiled code.
I can also give it a float stop value, in which case it will give me a float array - the default float type.
In [615]: np.arange(5.0)
Out[615]: array([ 0., 1., 2., 3., 4.])
In [616]: _.dtype
Out[616]: dtype('float64')
zeros is similar; the default dtype is float64, but with a parameter I can change that. Since its primary task with to allocate memory, and it doesn't have to do any calculation, I'm sure it creates the desired dtype right away, without further conversion. But again, this is compiled code, and I shouldn't have to worry about what it is doing under the covers.
In [618]: np.zeros(5)
Out[618]: array([ 0., 0., 0., 0., 0.])
In [619]: _.dtype
Out[619]: dtype('float64')
In [620]: np.zeros(5,dtype=np.float32)
Out[620]: array([ 0., 0., 0., 0., 0.], dtype=float32)
randn involves a lot of calculation, and evidently it is compiled to work with the default float type. It does not take a dtype. But since the result is an array, it can be cast with astype.
In [623]: np.random.randn(3)
Out[623]: array([-0.64520949, 0.21554705, 2.16722514])
In [624]: _.dtype
Out[624]: dtype('float64')
In [625]: __.astype(np.float32)
Out[625]: array([-0.64520949, 0.21554704, 2.16722512], dtype=float32)
Let me stress that astype is a method of an array. It takes the values of the array and produces a new array with the desire dtype. It does not act retroactively (or in-place) on the array itself, or on the function that created that array.
The effect of astype is often (always?) the same as a dtype parameter, but the sequence of actions is different.
In https://stackoverflow.com/a/39625960/901925 I describe a sparse matrix creator that takes a dtype parameter, and implements it with an astype method call at the end.
When you do calculations such as dot or *, it tries to match the output dtype with inputs. In the case of mixed types it goes with the higher precision alternative.
In [642]: np.arange(5,dtype=np.float32)*np.arange(5,dtype=np.float64)
Out[642]: array([ 0., 1., 4., 9., 16.])
In [643]: _.dtype
Out[643]: dtype('float64')
In [644]: np.arange(5,dtype=np.float32)*np.arange(5,dtype=np.float32)
Out[644]: array([ 0., 1., 4., 9., 16.], dtype=float32)
There are casting rules. One way to look those up is with can_cast function:
In [649]: np.can_cast(np.float64,np.float32)
Out[649]: False
In [650]: np.can_cast(np.float32,np.float64)
Out[650]: True
It is possible in some calculations that it will cast the 32 to 64, do the calculation, and then cast back to 32. The purpose would be to avoid rounding errors. But I don't know how you find that out from the documentation or tests.
arr1 = np.array([25, 56, 12, 85, 34, 75])
arr2 = np.array([42, 3, 86, 32, 856, 46])
arr1.astype(np.complex)
print (arr1)
print(type(arr1[0]))
print(arr1.astype(np.complex))
arr2 = np.array(arr2,dtype='complex')
print(arr2)
print(type(arr2[0]))
OUTPUT for above
[25 56 12 85 34 75]
<class 'numpy.int64'>
[25.+0.j 56.+0.j 12.+0.j 85.+0.j 34.+0.j 75.+0.j]
[ 42.+0.j 3.+0.j 86.+0.j 32.+0.j 856.+0.j 46.+0.j]
<class 'numpy.complex128'>
It can be seen that astype changes the type temporally as we do in normal type casting but where as the generic method changes the type permanently
.astype() copies the data.
>>> a = np.ones(3, dtype=float)
>>> a
array([ 1., 1., 1.])
>>> b = a.astype(int)
>>> b
array([1, 1, 1])
>>> np.may_share_memory(a, b)
False
Note that astype() copies the data even if the dtype is actually the same:
>>> c = a.astype(float)
>>> np.may_share_memory(a, c)
False