matrice1 = temp.unsqueeze(0)
print(M.shape)
matrice2 = M.permute(1, 0, 2, 3)
print(matrice2.shape)
print( torch.abs(matrice1 - matrice2).shape )
#torch.Size([1, 10, 3, 256])
#torch.Size([10, 1, 3, 256])
#torch.Size([10, 10, 3, 256])
I got the outcome above. I am wondering why the subtraction between two different dimension tensors make the outcome the tensor that has the shape as [10,10,3,256].
According to the broadcast semantics of PyTorch,The two tensors are "broadcastable" in your case, so they are automatically expanded to the equal size of torch.Size([10, 10, 3, 256]).
Related
I have an immature question.
For example, I got a tensor with the size of: torch.Size([2, 1, 80, 64]).
I need to turn it into another tensor with the size of: torch.Size([2, 1, 80, 16]).
Are there any right ways to achieve that?
There exist many functions to achieve dimensionality reduction and the following are some examples:
randomly select 16 out of the 64 features
take the mean of every four features (64/4=16)
use a dimensionality reduction technique like PCA
apply a linear transformation
apply a convolution function
To give a satisfying answer, more information about why and what you want to do is necessary.
Answered by: #ptrblck_de
slice the tensor
y = x[..., :16]
print(y.shape)
# torch.Size([2, 1, 80, 16])
index it with a stride of 4
y = x[..., ::4]
print(y.shape)
# torch.Size([2, 1, 80, 16])
use any pooling (max, avg, etc.) layer (the same would also work using adaptive pooling layers)
pool = nn.MaxPool2d((1, 2), (1, 4))
y = pool(x)
print(y.shape)
# torch.Size([2, 1, 80, 16])
pool = nn.AdaptiveAvgPool2d(output_size=(80, 16))
y = pool(x)
print(y.shape)
# torch.Size([2, 1, 80, 16])
or manually reduce the last dimension with any reduction op (sum, mean, max, etc.)
I am trying to use the permute function to swap the axis of my tensor but for some reason the output is not as expected. The output of the code is torch.Size([512, 256, 3, 3]) but I would expect it to be torch.Size([256, 512, 3, 3]). It doesn't look like I can use flip to switch 0, 1 index. Is there something i am missing? I wish to change the tensor such that the shape is (256, 512, 3, 3).
Reproducible code:
import torch
wtf = torch.rand(3, 3, 512, 256)
wtf = wtf.permute(2, 3, 1, 0)
print(wtf.shape)
The numbers provided to torch.permute are the indices of the axis in the order you want the new tensor to have.
Having set x as torch.rand(3, 3, 512, 256).
If you want to invert the order of axis: the initial order is 0, 1, 2, 3, you want 3, 2, 1, 0:
>>> wtf.permute(3, 2, 1, 0).shape
torch.Size([256, 512, 3, 3])
Inverting axis order is essentially the transpose operation:
>>> wtf.T
torch.Size([256, 512, 3, 3])
If you just want to invert and keep the order of the last two: original order is 0, 1, 2, 3 and resulting order is 3, 2, 0, 1:
>>> x.permute(3, 2, 0, 1).shape
torch.Size([256, 512, 3, 3])
The difference between the two options is that the last two axes of size 3 will be swapped.
I have a tensor Aof sizetorch.Size([32, 32, 3, 3]) and I want to split it and extract a tensor B of size torch.Size([16, 16, 3, 3]) from it. The tensor can be 1d or 4d and split has to be according to the given new tensor dimensions. I have been able to generate the target dimensions but I'm unable to split and extract the values from the source tensor. I ave tried torch.narrow but it takes only 3 arguments and I need 4 in many cases. torch.split takes dim as an int due to which tensor is split along one dimension only. But I want to split it along multiple dimensions.
You have multiple options:
use .split multiple times
use .narrow multiple times
use slicing
e.g.:
t = torch.rand(32, 32, 3, 3)
t0, t1 = t.split((16, 16), 0)
print(t0.shape, t1.shape)
>>> torch.Size([16, 32, 3, 3]) torch.Size([16, 32, 3, 3])
t00, t01 = t0.split((16, 16), 1)
print(t00.shape, t01.shape)
>>> torch.Size([16, 16, 3, 3]) torch.Size([16, 16, 3, 3])
t00_alt, t01_alt = t[:16, :16, :, :], t[16:, 16:, :, :]
print(t00_alt.shape, t01_alt.shape)
>>> torch.Size([16, 16, 3, 3]) torch.Size([16, 16, 3, 3])
I have the following tensor :
ts = torch.tensor([[1,2,3],[4,6,7],[8,9,10]])
> tensor([[ 1, 2, 3],
[ 4, 6, 7],
[ 8, 9, 10]])
I am looking for a pytorch generic operation that adds all rows element-wise like that:
ts2 = ts[0]+ts[1]+ts[2]
print(ts2)
> tensor([13, 17, 20])
In reality, the number of rows corresponds to the batch size that vary.
You can sum over an axis/dimension like so:
torch.sum(ts, dim=0)
I am trying to extract the unique values in each row of a matrix and returning them into the same matrix (with repeated values set to say, 0) For example, I would like to transform
torch.Tensor(([1, 2, 3, 4, 3, 3, 4],
[1, 6, 3, 5, 3, 5, 4]])
to
torch.Tensor(([1, 2, 3, 4, 0, 0, 0],
[1, 6, 3, 5, 0, 0, 4]])
or
torch.Tensor(([1, 2, 3, 4, 0, 0, 0],
[1, 6, 3, 5, 4, 0, 0]])
I.e. the order does not matter in the rows. I have tried using pytorch.unique() and in the documentation it is mentioned that the dimension to take the unique values can be specified with the parameter dim. However, It doesn't seem to work for this case.
I've tried:
output= torch.unique(torch.Tensor([[4,2,52,2,2],[5,2,6,6,5]]), dim = 1)
output
Which gives
tensor([[ 2., 2., 2., 4., 52.],
[ 2., 5., 6., 5., 6.]])
Does anyone have a particular fix for this? If possible, I'm trying to avoid for loops.
One must admit the unique function can sometimes be very confusing without given proper examples and explanations.
The dim parameter specifies which dimension on the matrix tensor you want to apply on.
For instance, in a 2D matrix, dim=0 will let operation perform vertically where dim=1 means horizontally.
Example, let's consider a 4x4 matrix with dim=1. As you can see from my code below, the unique operation is applied row by row.
You notice the double occurrence of the number 11 in the first and last row. Numpy and Torch does this to preserve the shape of the final matrix.
However, if you do not specify any dimension, torch will automatically flatten your matrix and then apply unique to it and you will get a 1D array that contains unique data.
import torch
m = torch.Tensor([
[11, 11, 12,11],
[13, 11, 12,11],
[16, 11, 12, 11],
[11, 11, 12, 11]
])
output, indices = torch.unique(m, sorted=True, return_inverse=True, dim=1)
print("Ori \n{}".format(m.numpy()))
print("Sorted \n{}".format(output.numpy()))
print("Indices \n{}".format(indices.numpy()))
# without specifying dimension
output, indices = torch.unique(m, sorted=True, return_inverse=True)
print("Sorted (no dim) \n{}".format(output.numpy()))
Result (dim=1)
Ori
[[11. 11. 12. 11.]
[13. 11. 12. 11.]
[16. 11. 12. 11.]
[11. 11. 12. 11.]]
Sorted
[[11. 11. 12.]
[11. 13. 12.]
[11. 16. 12.]
[11. 11. 12.]]
Indices
[1 0 2 0]
Result (no dimension)
Sorted (no dim)
[11. 12. 13. 16.]
I was confused when using torch.unique the first time. After doing some experiments I have finally figured out how the dim argument works.
Docs of torch.unique says that:
counts (Tensor): (optional) if return_counts is True, there will be an additional returned tensor (same shape as output or output.size(dim), if dim was specified) representing the number of occurrences for each unique value or tensor.
For example, if your input tensor is a 3D tensor of size n x m x k and dim=2, unique will work separately on k matrices of size n x m. In other words, it will treat all dimensions other than the dim 2 as a single tensor.