I am trying to extract the unique values in each row of a matrix and returning them into the same matrix (with repeated values set to say, 0) For example, I would like to transform
torch.Tensor(([1, 2, 3, 4, 3, 3, 4],
[1, 6, 3, 5, 3, 5, 4]])
to
torch.Tensor(([1, 2, 3, 4, 0, 0, 0],
[1, 6, 3, 5, 0, 0, 4]])
or
torch.Tensor(([1, 2, 3, 4, 0, 0, 0],
[1, 6, 3, 5, 4, 0, 0]])
I.e. the order does not matter in the rows. I have tried using pytorch.unique() and in the documentation it is mentioned that the dimension to take the unique values can be specified with the parameter dim. However, It doesn't seem to work for this case.
I've tried:
output= torch.unique(torch.Tensor([[4,2,52,2,2],[5,2,6,6,5]]), dim = 1)
output
Which gives
tensor([[ 2., 2., 2., 4., 52.],
[ 2., 5., 6., 5., 6.]])
Does anyone have a particular fix for this? If possible, I'm trying to avoid for loops.
One must admit the unique function can sometimes be very confusing without given proper examples and explanations.
The dim parameter specifies which dimension on the matrix tensor you want to apply on.
For instance, in a 2D matrix, dim=0 will let operation perform vertically where dim=1 means horizontally.
Example, let's consider a 4x4 matrix with dim=1. As you can see from my code below, the unique operation is applied row by row.
You notice the double occurrence of the number 11 in the first and last row. Numpy and Torch does this to preserve the shape of the final matrix.
However, if you do not specify any dimension, torch will automatically flatten your matrix and then apply unique to it and you will get a 1D array that contains unique data.
import torch
m = torch.Tensor([
[11, 11, 12,11],
[13, 11, 12,11],
[16, 11, 12, 11],
[11, 11, 12, 11]
])
output, indices = torch.unique(m, sorted=True, return_inverse=True, dim=1)
print("Ori \n{}".format(m.numpy()))
print("Sorted \n{}".format(output.numpy()))
print("Indices \n{}".format(indices.numpy()))
# without specifying dimension
output, indices = torch.unique(m, sorted=True, return_inverse=True)
print("Sorted (no dim) \n{}".format(output.numpy()))
Result (dim=1)
Ori
[[11. 11. 12. 11.]
[13. 11. 12. 11.]
[16. 11. 12. 11.]
[11. 11. 12. 11.]]
Sorted
[[11. 11. 12.]
[11. 13. 12.]
[11. 16. 12.]
[11. 11. 12.]]
Indices
[1 0 2 0]
Result (no dimension)
Sorted (no dim)
[11. 12. 13. 16.]
I was confused when using torch.unique the first time. After doing some experiments I have finally figured out how the dim argument works.
Docs of torch.unique says that:
counts (Tensor): (optional) if return_counts is True, there will be an additional returned tensor (same shape as output or output.size(dim), if dim was specified) representing the number of occurrences for each unique value or tensor.
For example, if your input tensor is a 3D tensor of size n x m x k and dim=2, unique will work separately on k matrices of size n x m. In other words, it will treat all dimensions other than the dim 2 as a single tensor.
Related
What I have is the following tensor:
a = np.array([[1,1], [2,2], [3,3]])
t = torch.from_numpy(a)
I need an operation that gets me to the following matrix:
enter image description here
This matrix contains the element-wise dot product of the tensor if it gets multiplied by its transposed version, i.e., the first element on the diagonal is
1x1 + 1x1 = 2, the 2nd element on the diagonal is
2x2 + 2x2 = 8, and the 2nd element in the first column is 1x2 + 1x2 = 4,
and so on. How do I get this matrix in torch, starting from tensor t? Thank you!
I tried various combinations of torch.tensordot, torch.transpose, etc.
einsum it!
print(torch.einsum("ik,jk->ij", t, t))
print(torch.einsum("ki,kj->ij", t, t))
tensor([[ 2, 4, 6],
[ 4, 8, 12],
[ 6, 12, 18]], dtype=torch.int32)
tensor([[14, 14],
[14, 14]], dtype=torch.int32)
I have two tensors a and b. And I want to retrive the values of b according to the positions of max values in a. That is,
max_values, indices = torch.max(a, dim=0, keepdim=True)
However, I do not know how to use the indices to retrive the values of b. Can anybody helps to solve it? Thanks a lot!!
Edit:
Sorry for not describing my problem concretely. To give a minimal example, the value of tensors a and b are:
a = torch.tensor([[1,2,4],[2,1,3]])
b = torch.tensor([[10,24,2],[23,4,5]])
If I use torch.max(a, dim=0, keepdim=True), it will return:
max: tensor([[2, 2, 4]])
indices: tensor([[1, 0, 0]])
What I want to obtain is the selected value of tensor b according to the indices of max values of a in dim=0, that is,
tensor([[23, 24, 2]])
I have tried b[indices], whereas the result is not what I want:
tensor([[[ 2, 3, 5],
[10, 30, 40],
[10, 30, 40]]])
You can use torch.gather:
torch.gather(b, dim=0, index=indices)
Suppose I have the following two matching shape tensors:
a = tensor([[ 0.0113, -0.1666, 0.5960, -0.0667], [-0.0977, -0.1984, 0.5153, 0.0420]])
selectors = tensor([[ True, True, False, False], [ True, False, True, False]])
When using torch.masked_select to find the values in a that match True indices in selectors like this:
torch.masked_select(a, selectors)
The output will be in 1D shape instead of the original 2D shape:
tensor([ 0.0113, -0.1666, -0.0977, 0.5153])
This is consistent with masked_select behavior as it is given in the documentation (torch.masked_select). However, my goal is to get a result that matches the shape of the two original tensors. I.e.:
tensor([[0.0113, -0.1666], [-0.0977, 0.5153]])
Is there a way to get this without having to loop over all the elements in the tensors and find the mask for each one? Please note that I have also looked into using torch.where, but it doesn't fit the case I have as I see it.
As #jodag pointed out, for general inputs, each row on the desired masked result might have a different number of elements, depending on how many True values there are on the same row in selectors. However, you could overcome this by allowing trailing zero padding in the result.
Basic solution:
indices = torch.masked_fill(torch.cumsum(selectors.int(), dim=1), ~selectors, 0)
masked = torch.scatter(input=torch.zeros_like(a), dim=1, index=indices, src=a)[:,1:]
Explanation:
By applying cumsum() row-wise over selectors, we compute for each unmasked element in a the target column number it should be copied to in the output tensor. Then, scatter() performs a row-wise scattering of a's elements to these computed target locations. We leave all masked elements with the index 0, so that the first element in each row of the result would contain one of the masked elements (maybe arbitrarily. we don't care which). We then ignore these un-wanted 1st values by taking the slice [:,1:]. The output resulting masked tensor has the exact same size as the input a (this is the maximum needed size, for the case where there is a row of full True values in selectors).
Usage example:
>>> a = Torch.tensor([[ 1, 2, 3, 4, 5, 6], [10, 20, 30, 40, 50, 60]])
>>> selectors = Torch.tensor([[ True, False, False, True, False, True], [False, False, True, True, False, False]])
>>> torch.cumsum(selectors.int(), dim=1)
tensor([[1, 1, 1, 2, 2, 3],
[0, 0, 1, 2, 2, 2]])
>>> indices = torch.masked_fill(torch.cumsum(selectors.int(), dim=1), ~selectors, 0)
>>> indices
tensor([[1, 0, 0, 2, 0, 3],
[0, 0, 1, 2, 0, 0]])
>>> torch.scatter(input=torch.zeros_like(a), dim=1, index=indices, src=a)
tensor([[ 5, 1, 4, 6, 0, 0],
[60, 30, 40, 0, 0, 0]])
>>> torch.scatter(input=torch.zeros_like(a), dim=1, index=indices, src=a)[:,1:]
tensor([[ 1, 4, 6, 0, 0],
[30, 40, 0, 0, 0]])
Adapting output size: Here, the length of dim=1 of the output resulting masked tensor is the max number of un-masked items in a row. For your original show-case, the output shape would be (2,2) as you desired. Note that if this number is not previously known and a is on CUDA, it would cause an additional host-device synchronization that might affect the performance.
To do so, instead of allocating input=torch.zeros_like(a) for scatter(), allocate it by a.new_zeros(size=(a.size(0), torch.max(indices).item() + 1)). The +1 is for the 1st place which is later sliced-out. The host-device synchronization would occur by accessing the result of max() to calculate the allocated output size.
Example:
>>> torch.scatter(input=a.new_zeros(size=(a.size(0), torch.max(indices).item() + 1)), dim=1, index=indices, src=a)[:,1:]
tensor([[ 1, 4, 6],
[30, 40, 0]])
Changing the padding value: If another custom default value is wanted as a padding, one could use torch.full_like(my_custom_value) rather than torch.zeros_like() when allocating the output for scatter().
I am trying to use numpy to multiply two matrices:
import numpy as np
A = np.array([[1, 3, 2], [4, 0, 1]])
B = np.array([[1, 0, 5], [3, 1, 2]])
I tested the process and ran the calculations manually, utilizing the formula for matrix multiplications. So, in this case, I would first multiply [1, 0, 5] x A, which resulted in [11, 9] and then multiply [3, 1, 2] x B, which resulted in [10, 14]. Finally, the product of this multiplication is [[11, 9], [10, 14]]
nevertheless, when I use numpy to multiply these matrices, I am getting an error:
ValueError: shapes (2,3) and (2,3) not aligned: 3 (dim 1) != 2 (dim 0)
Is there a way to do this with python, successfully?
Read the docs on matrix multiplication in numpy, specifically on behaviours.
The behavior depends on the arguments in the following way.
If both arguments are 2-D they are multiplied like conventional
matrices. If either argument is N-D, N > 2, it is treated as a stack
of matrices residing in the last two indexes and broadcast
accordingly. If the first argument is 1-D, it is promoted to a matrix
by prepending a 1 to its dimensions. After matrix multiplication the
prepended 1 is removed. If the second argument is 1-D, it is promoted
to a matrix by appending a 1 to its dimensions. After matrix
multiplication the appended 1 is removed.
to get your output, try transposing one before multiplying?
c=np.matmul(A,B.transpose())
array([[11, 10],
[ 9, 14]])
I have the following tensor :
ts = torch.tensor([[1,2,3],[4,6,7],[8,9,10]])
> tensor([[ 1, 2, 3],
[ 4, 6, 7],
[ 8, 9, 10]])
I am looking for a pytorch generic operation that adds all rows element-wise like that:
ts2 = ts[0]+ts[1]+ts[2]
print(ts2)
> tensor([13, 17, 20])
In reality, the number of rows corresponds to the batch size that vary.
You can sum over an axis/dimension like so:
torch.sum(ts, dim=0)