I have a 1d-numpy array that I want to split to number of sub arrays of fixed length (note it len).
When the length of the array is not an multiply of len ,I want it to take the last len elements of the array.
When using the numpy function array_split we get:
x = np.arange(8.0)
print(np.array_split(x, 3))
The output is:
[array([0., 1., 2.]), array([3., 4., 5.]), array([6., 7.])]
But I want:
[array([0., 1., 2.]), array([3., 4., 5.]), array([5., 6., 7.])]
Does anyone know a built-in function that does that?
Thnx!
I think this is probably your best bet -
import numpy as np
x = np.arange(8)
c = 3
split = np.array_split(x, c)[:-1] + [x[-int(round(len(x)/c)):]]
print(split)
Gives -
[array([0, 1, 2]), array([3, 4, 5]), array([5, 6, 7])]
Using a lenght parameter and list comprehensions:
import numpy as np
x = np.arange(8)
lenght=3
split=[x[i:i+lenght] for i in range(0,len(x),lenght)][:-1]+[x[-lenght:]]
print(split)
*(do not use len keyword, as it is also a python's built-in)
Related
I have a tensor t of dim n x 3. When I apply torch.linalg.norm it returns one single value. What I need is a batch-wise norm function which will return a tensor with n norms, one for each vector in t.
Thanks for your help.
It seems the most relevant documentation place is:
https://pytorch.org/docs/stable/generated/torch.linalg.norm.html
In the terminal you could try: python3 and then the following python commands:
>>> from torch import linalg as LA
>>> c = torch.tensor([[1., 2., 3.],
... [-1, 1, 4]])
>>> LA.norm(c, dim=0)
tensor([1.4142, 2.2361, 5.0000])
>>> LA.norm(c, dim=1)
tensor([3.7417, 4.2426])
Conclusion:
In your specific case you will need to do:
torch.linalg.norm(t,dim=1)
In general, my question is on the possible ways for creating/appending to nested structured arrays. Specifically where the dtype structure is known, but the size in the nested elements weren't pre-defined. I have tried different things with the join_by and fromarray methods, but haven't had success. The doc examples seem to suggest the join type methods are useful for a sort of 'zipping' of data and not really extending it?
Datatypes defined
from numpy.lib import recfunctions as rfn
c = [('x','f8'),('y','f8')]
d = [('text','|S20'),('val', c)]
Creates 2 rows of 'd'
zeros = np.zeros(2, dtype=d)
print(f'{zeros=}')
How to make multiple rows of 'c'?
c = [('x','f8'),('y','f8')]
# Add a number to the construct for 'c'
d = [('text','|S20'),('val', c, 3)]
zeros = np.zeros(2, dtype=d)
print(f'{zeros=}')
How to do this without predefining the size of 'c'?
Appending/Extending nested elements doesn't seem to work
At the lowest level, 'c' can be used to construct a recarray
large_c = np.arange(2*3).reshape(3,2)
struct_c = rfn.unstructured_to_structured(large_c, np.dtype(c))
print(f'{struct_c=}')
But now, is there a way to construct the next level from existing structured arrays?
I thought constructing an array with a shape that seems to broadcast would work, but it fails
struct_d = rfn.unstructured_to_structured(np.array(['', large_c]), np.dtype(d))
print(f'{struct_d=}')
>> The length of the last dimension of arr must be equal to the number of fields in dtype
I also tried the recarray constructions, which had their own issues
struct_d = np.core.records.fromarrays(np.array(['', large_c]), dtype=d)
print(f'{struct_d=}')
>>> array-shape mismatch in array 1
I think the most straightforward way of making such an array is:
In [352]: c = [('x','f8'),('y','f8')]
...: # Add a number to the construct for 'c'
...: d = [('text','|S20'),('val', c, 3)]
...: zeros = np.zeros(2, dtype=d)
In [353]: zeros
Out[353]:
array([(b'', [(0., 0.), (0., 0.), (0., 0.)]),
(b'', [(0., 0.), (0., 0.), (0., 0.)])],
dtype=[('text', 'S20'), ('val', [('x', '<f8'), ('y', '<f8')], (3,))])
In [355]: x = np.arange(12).reshape(2,3,2)
In [357]: carr = rf.unstructured_to_structured(x,np.dtype(c))
In [358]: carr
Out[358]:
array([[( 0., 1.), ( 2., 3.), ( 4., 5.)],
[( 6., 7.), ( 8., 9.), (10., 11.)]],
dtype=[('x', '<f8'), ('y', '<f8')])
With the right shape and dtype, it can be assigned to the larger array:
In [359]: zeros['val']=carr
In [360]: zeros
Out[360]:
array([(b'', [( 0., 1.), ( 2., 3.), ( 4., 5.)]),
(b'', [( 6., 7.), ( 8., 9.), (10., 11.)])],
dtype=[('text', 'S20'), ('val', [('x', '<f8'), ('y', '<f8')], (3,))])
I'm not sure if there's something in your question that I glossed over or not.
This question already has answers here:
How do I add an extra column to a NumPy array?
(17 answers)
Closed 5 years ago.
l have the following vector
video_132.shape
Out[64]: (64, 3)
that l would to add to it a new 3D vector of three values
video_146[1][146][45]
such that
video_146[1][146][45].shape
Out[68]: (3,)
and
video_146[1][146][45]
Out[69]: array([217, 207, 198], dtype=uint8)
when l do the following
np.append(video_132,video_146[1][146][45])
l'm supposed to get
video_132.shape
Out[64]: (65, 3) # originally (64,3)
However l get :
Out[67]: (195,) # 64*3+3=195
It seems that it flattens the vector
How can l do the append by preserving the 3D structure ?
For visual simplicity let's rename video_132 --> a, and video_146[1][146][45] --> b. The particular values aren't important so let's say
In [82]: a = np.zeros((64, 3))
In [83]: b = np.ones((3,))
Then we can append b to a using:
In [84]: np.concatenate([a, b[None, :]]).shape
Out[84]: (65, 3)
Since np.concatenate returns a new array, reassign its return value to a to "append" b to a:
a = np.concatenate([a, b[None, :]])
Code for append:
def append(arr, values, axis=None):
arr = asanyarray(arr)
if axis is None:
if arr.ndim != 1:
arr = arr.ravel()
values = ravel(values)
axis = arr.ndim-1
return concatenate((arr, values), axis=axis)
Note how arr is raveled if no axis is provided
In [57]: np.append(np.ones((2,3)),2)
Out[57]: array([1., 1., 1., 1., 1., 1., 2.])
append is really aimed as simple cases like adding a scalar to a 1d array:
In [58]: np.append(np.arange(3),6)
Out[58]: array([0, 1, 2, 6])
Otherwise the behavior is hard to predict.
concatenate is the base operation (builtin) and takes a list, not just two. So we can collect many arrays (or lists) in one list and do one concatenate at the end of a loop. And since it doesn't tweak the dimensions before hand, it forces us to do that ourselves.
So to add a shape (3,) to a (64,3) we have transform that (3,) into (1,3). append requires the same dimension adjustment as concatenate if we specify the axis.
In [68]: np.append(arr,b[None,:], axis=0).shape
Out[68]: (65, 3)
In [69]: np.concatenate([arr,b[None,:]], axis=0).shape
Out[69]: (65, 3)
I'm trying to plot a numpy array with shape [height x width x 3] containing RGB values. As a simple example suppose you have the Belgian flag:
import numpy as np
import matplotlib.pyplot as plt
flag = np.empty((1,3,3))
flag[0,0,:] = (0,0,0)
flag[0,1,:] = (254,240,71)
flag[0,2,:] = (255,55,14)
plt.imshow(flag)
plt.show()
This results in the following output:
Can anyone tell me why it is not plotting the right RGB values? Did I make a mistake in the dimensionality? Probably an easy answer to this, but can't seem to find it .. Thanks a lot for any advice!
The default data type for the array created by numpy.empty is floating point, and imshow treats floating point values differently than integer values. (imshow expects floating point values to be in the range 0.0 to 1.0.)
Change this
flag = np.empty((1,3,3))
to
flag = np.empty((1,3,3), dtype=np.uint8)
The reason you got those particular colors when flag is floating point is that imshow apparently converted your array to integer without checking that the input values were in the range 0.0 to 1.0. Here's what happens in that case:
In [25]: flag
Out[25]:
array([[[ 0., 0., 0.],
[ 254., 240., 71.],
[ 255., 55., 14.]]])
In [26]: img = (flag*255).astype(np.uint8)
In [27]: img
Out[27]:
array([[[ 0, 0, 0],
[ 2, 16, 185],
[ 1, 201, 242]]], dtype=uint8)
If you then run imshow(img), you get the black, blue and cyan plot.
Try to use the float values between 0 ~ 1.
So change the code like this,
flag[0,0,:] = (0,0,0)
flag[0,1,:] = (254/255,240/255,71/255)
flag[0,2,:] = (255/255,55/255,14/255)
In the Theano Tutorial:
http://deeplearning.net/software/theano/tutorial/gradients.html
or
>>> import theano
>>> import theano.tensor as T
>>> x = T.dvector('x')
>>> y = x ** 2
>>> J, updates = theano.scan(lambda i, y,x : T.grad(y[i], x), sequences=T.arange(y.shape[0]), non_sequences=[y,x])
>>> f = theano.function([x], J, updates=updates)
>>> f([4, 4])
array([[ 8., 0.],
[ 0., 8.]])
"theano.scan" is used to compute the Jacobian. I understand how to compute the Jacobian by hand but I do not quite understand how the function scan is calculating it iteratively; specifically, I don't understand how the "zeroes" are obtained in the Jacobian.
Can someone show me, for each iteration, how this is calculated? Thanks!