I'm struggling to understand what the function torch.gather is doing.
If I have:
a = torch.tensor([[4,5,6],[7,8,9],[10,11,12]])
b = torch.tensor([[1,1],[1,2]])
Then
>>> torch.gather(input=a, dim=1, index=b)
tensor([[5, 5],
[8, 9]])
While
>>> torch.gather(input=a, dim=0, index=b)
tensor([[ 7, 8],
[ 7, 11]])
Can somebody explain very simply how these output tensors are actually constructed?
On the first call you index a with b along dim=1 (2nd dimension). The performed operation is:
out[i,j] = a[i, b[i,j]]
Which returns:
[[ a[0, b[0,0]], a[0, b[0,1]] ],
[ a[1, b[1,0]], a[1, b[1,1]] ]]
While on the second call, you index a with b along dim=0 (1st dimension). This time around:
out[i,j] = a[b[i,j], j]
Which returns:
[[ a[b[0,0], 0], a[b[0,1], 1] ],
[ a[b[1,0], 0], a[b[1,1], 1] ]]
For more explanations, read on with this thread.
Related
I have a huge torch Tensor and I'm looking for an efficient approach to subtract the elements of every pair of that Tensor.
Of course I could use two nested for but it wouldn't be efficient.
For example giving
[1, 2, 3, 4]
The output I want is
[1-2, 1-3, 1-4, 2-3, 2-4, 3-4]
You can do this easily:
>>> x = torch.tensor([1, 2, 3, 4])
>>> x[:, None] - x[None, :]
tensor([[ 0, -1, -2, -3],
[ 1, 0, -1, -2],
[ 2, 1, 0, -1],
[ 3, 2, 1, 0]])
see more details here.
Sorry if this is a noob question.
i have a list, for exemple "a" which has two or tree list inside a1,a2,a3 like :
[ [1,2], [0,0,0], [1,2,3,4,5] ]
[ [1,2,3,4], [0,0], [1,2,3] ]
etc...
Each list inside doesn't has the same length so when I make a simple
for i in range(len(a)):
print(a[i])
the result is :
[[1,2],[0,0,0],[1,2,3,4,5]]
[[1,2,3,4],[0,0],[1,2,3]]
While I'm waiting for this result where elements in list are aligned in columns :
[[1,2] [0,0,0] [1,2,3,4,5]]
[[1,2,3,4] [0,0] [1,2,3]]
How can I do that?
thanks in advance for any help.
You can use string's ljust method to align the lists according to the largest:
a = [[[1, 2], [0, 0, 0], [1, 2, 3, 4, 5]],
[[1, 2, 3, 4], [0, 0], [1, 2, 3]]]
size = max([len(str(x)) for sub in a for x in sub]) + 4
for sub in a:
for x in sub:
print(str(x).ljust(size), end="")
print()
This Gives:
[1, 2] [0, 0, 0] [1, 2, 3, 4, 5]
[1, 2, 3, 4] [0, 0] [1, 2, 3]
Another option using one loop and the join method could be:
for sub in a:
print(''.join((str(x).ljust(size) for x in sub)))
Another quick-and-dirty approach, using str.format:
a = [[ [1,2], [0,0,0], [1,2,3,4,5] ],
[ [1,2,3,4], [0,0], [1,2,3] ],
[ [1], [2,2]],
[]]
max_sublist = max(a, key=len)
max_len = len(str(max((i for subl in a for i in subl), key=lambda k: len(str(k)))))
for subl in a:
print('[{}]'.format(('{!s:<{width}}' * len(max_sublist)).format(*subl, *' '*len(max_sublist), width=max_len)))
Prints:
[[1, 2] [0, 0, 0] [1, 2, 3, 4, 5]]
[[1, 2, 3, 4] [0, 0] [1, 2, 3] ]
[[1] [2, 2] ]
[ ]
Assume that there is a list of small tensors (say 16 blocks), and it is desired to stick these small tensors along horizontally and vertically to create a larger 2D image.
torch.split() is available for partition the tensor to smaller blocks, is there any operation for the opposite case?
Thanks
You are looking for torch.cat with dim. To stack tensors vertically use torch.cat(..., dim=0), to stack horizontally use torch.cat(..., dim=1):
Example:
tensors = torch.split(torch.randn(4, 6), 2, dim=1)
tensors
(tensor([[-1.0257, 0.5213],
[-0.1181, -1.4420],
[-1.5563, -1.0757],
[ 1.1788, 0.6222]]), tensor([[-0.4531, -0.1260],
[-0.2383, -1.3542],
[-0.8752, -0.4728],
[ 0.7879, 1.3686]]), tensor([[ 2.3357, -0.6220],
[ 0.2687, 0.1146],
[ 0.9912, -0.0586],
[-0.8507, 0.5126]]))
Stack vertically along first dimension:
torch.cat(tensors, dim=0)
tensor([[-1.0257, 0.5213],
[-0.1181, -1.4420],
[-1.5563, -1.0757],
[ 1.1788, 0.6222],
[-0.4531, -0.1260],
[-0.2383, -1.3542],
[-0.8752, -0.4728],
[ 0.7879, 1.3686],
[ 2.3357, -0.6220],
[ 0.2687, 0.1146],
[ 0.9912, -0.0586],
[-0.8507, 0.5126]])
Stack horizontally along second dimension:
torch.cat(tensors, dim=1)
tensor([[-1.0257, 0.5213, -0.4531, -0.1260, 2.3357, -0.6220],
[-0.1181, -1.4420, -0.2383, -1.3542, 0.2687, 0.1146],
[-1.5563, -1.0757, -0.8752, -0.4728, 0.9912, -0.0586],
[ 1.1788, 0.6222, 0.7879, 1.3686, -0.8507, 0.5126]])
Building block matrices can be done with a bit of fiddling with matrix operations. Lets say you have 4 matrices A, B, C and D you want to stack into
A B
C D
a = torch.tensor([[1,2],[3,4]])
tensor([[1, 2],
[3, 4]])
b = torch.tensor([[5,6],[7,8]])
tensor([[5, 6],
[7, 8]])
c = torch.tensor([[-1,-2],[-3,-4]])
tensor([[-1, -2],
[-3, -4]])
d = torch.tensor([[-5,-6],[-7,-8]])
tensor([[-5, -6],
[-7, -8]])
Then concatenate, transpose and break into two chunks. (Transpose is needed because we will transpose later, and this one cancels out).
x,y = torch.cat((a,b,c,d),dim=1).t().chunk(2)
(tensor([[1, 3],
[2, 4],
[5, 7],
[6, 8]]),
tensor([[-1, -3],
[-2, -4],
[-5, -7],
[-6, -8]]))
Next put those two matrices side-by-side and transpose
torch.cat((x,y),dim=1).t()
tensor([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[-1, -2, -5, -6],
[-3, -4, -7, -8]])
It should be straightforward to generalise to NxN block matrices.
Hi I have 2 arrays of vectors:
A=np.array([[5,62,7],[5,62,7],[5,62,7]])
B=np.array([[1,2,3],[1,2,3],[1,2,3]])
and I would like to concentrate them like that:
C=[[[5,62,7], [1,2,3]],
[[5,62,7], [1,2,3]],
[[5,62,7], [1,2,3]]]
The newish stack makes this easy:
In [130]: A=np.array([[5,62,7],[5,62,7],[5,62,7]])
...: B=np.array([[1,2,3],[1,2,3],[1,2,3]])
...:
In [131]: np.stack((A,B), axis=1)
Out[131]:
array([[[ 5, 62, 7],
[ 1, 2, 3]],
[[ 5, 62, 7],
[ 1, 2, 3]],
[[ 5, 62, 7],
[ 1, 2, 3]]])
It adds an extra dimension to each of the arrays, and then concatenates. With axis=0 is behave just like np.array.
np.array((A,B)).transpose(1,0,2)
joins them on a new 1st axis, and then moves it over.
hstack().reshape() to the rescue:
import numpy as np
A=np.array([[5,62,7],[5,62,7],[5,62,7]])
B=np.array([[1,2,3],[1,2,3],[1,2,3]])
c = np.hstack((A,B)).reshape(3,2,3)
print(c)
Output:
[[[ 5 62 7] [ 1 2 3]]
[[ 5 62 7] [ 1 2 3]]
[[ 5 62 7] [ 1 2 3]]]
hstack
reshape
I have one tensor in shape (2, G) and another in shape (N, 2).
I need to add them in such a way that the output is (N, 2, G), meaning that the first tensor is replicated to (N, 2, G) and then the second tensor is added to each matrix along the third dimension. (or vice versa: the second tensor is replicated to (N, 2, G) and the first one is added to every sub-tensor along the first dimension).
How can this done efficiently in Theano?
Thanks.
In an attempt to understand the problem, the following example is assumed to be representative.
If
A = [[1, 2, 3],
[4, 5, 6]]
and
B = [[1, 2],
[3, 4],
[5, 6],
[7, 8]]
then the result should be
C = [[[ 2. 3. 4.]
[ 6. 7. 8.]]
[[ 4. 5. 6.]
[ 8. 9. 10.]]
[[ 6. 7. 8.]
[ 10. 11. 12.]]
[[ 8. 9. 10.]
[ 12. 13. 14.]]]
Here G=3 and N=4.
To achieve this in Theano, one need only add new broadcastable dimensions and rely on broadcasting to get the desired result.
import numpy
import theano
import theano.tensor as tt
x = tt.matrix()
y = tt.matrix()
z = x.dimshuffle('x', 0, 1) + y.dimshuffle(0, 1, 'x')
f = theano.function([x, y], outputs=z)
print f(numpy.array([[1, 2, 3], [4, 5, 6]]), numpy.array([[1, 2], [3, 4], [5, 6], [7, 8]]))