Let's say I have a numpy array as such:
a = [0, 1, …, i-1, i, i+1, …, j, j+1, …, n]
and I'd like to replace of i-th, i+1-th… j-th element with a single one — their average:
b = [0, 1, …, i-1, average, j+1, …, n]
How would I do that with as compact code as possible?
Slice and concatenate arrays
np.concatenate([a[:i], a[i:j].mean().reshape(1,), a[j:]])
Example
a = np.array(list(range(20)))
i = 5
j = 10
np.concatenate([a[:i], a[i:j].mean().reshape(1,), a[j:]])
array([ 0., 1., 2., 3., 4., 7., 10., 11., 12., 13., 14., 15., 16.,
17., 18., 19.])
Related
Consider following matrices
>>> a = torch.Tensor([[1,2,3],[4,5,6], [7,8,9]])
>>> a
tensor([[1., 2., 3.],
[4., 5., 6.],
[7., 8., 9.]])
>>> b = torch.tensor([[1,1],[1,1]])
>>> b
tensor([[1, 1],
[1, 1]])
I want to replace 4 elements in a with b where their indices are specified in X = [0,2] and Y = [0,2]
To have:
>>>a
tensor([[1., 2., 1.],
[4., 5., 6.],
[1., 8., 1.]])
I look for some operations like scatter or put_index to update the matrix in few commands (not loops).
If we consider X and Y two tensors of horizontal and vertical indices, the following can work:
a[X.reshape(-1,1), Y] = b
To better understand how nn.BatchNorm2d works, I wanted to recreate the following lines of code:
input = torch.randint(1, 5, size=(2, 2, 3, 3)).float()
batch_norm = nn.BatchNorm2d(2)
output = (batch_norm(input))
print(input)
print(output)
tensor([[[[3., 3., 2.],
[3., 2., 2.],
[4., 2., 1.]],
[[1., 1., 2.],
[3., 2., 1.],
[1., 1., 1.]]],
[[[4., 3., 3.],
[4., 1., 4.],
[1., 3., 2.]],
[[2., 1., 4.],
[4., 2., 1.],
[4., 1., 3.]]]])
tensor([[[[ 0.3859, 0.3859, -0.6064],
[ 0.3859, -0.6064, -0.6064],
[ 1.3783, -0.6064, -1.5988]],
[[-0.8365, -0.8365, 0.0492],
[ 0.9349, 0.0492, -0.8365],
[-0.8365, -0.8365, -0.8365]]],
[[[ 1.3783, 0.3859, 0.3859],
[ 1.3783, -1.5988, 1.3783],
[-1.5988, 0.3859, -0.6064]],
[[ 0.0492, -0.8365, 1.8206],
[ 1.8206, 0.0492, -0.8365],
[ 1.8206, -0.8365, 0.9349]]]]
To achieve, I first calculated the mean and variance for each channel:
my_mean = (torch.mean(input, dim=[0, 2, 3]))
my_var = (torch.var(input, dim=[0, 2, 3]))
print(my_mean, my_var)
tensor([2.8333, 2.5556])
tensor([1.3235, 1.3203])
This seems reasonable, I have the mean and variance for each channel across the whole batch. Then I wanted to simply extract the mean from the input and divide by the variance. This is where problems arise, since I do not know to properly set up the mean and variance. PyTorch does not seem to broadcast properly:
my_output = (input - my_mean) / my_var
RuntimeError: The size of tensor a (3) must match the size of tensor b (2) at non-singleton dimension 3
I then wanted to reshape the mean and variance in the appropriate shape, such that each value is repeated 25 times in a 5x5 shape
First try:
my_mean.repeat(25).reshape(3, 5, 5)
But this also results in an error. What is the best way to achieve my goal?
I am looking to Concatenate 2 torch tensors at a certain index. As an example, I want to add b after a[1].
a = torch.Tensor([1, 2, 3, 4, 5])
b = torch.Tensor([6, 7, 8, 9, 10])
The desired output is
torch.Tensor([1, 2, 6, 7, 8, 9, 10, 3, 4, 5])
I tried torch.cat, but I can only have
tensor([ 6., 7., 8., 9., 10., 1., 2., 3., 4., 5.])
tensor([ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10.])
you will need to split the first tensor and concatenate the second in between
torch.cat([a[:2], b, a[2:]])
output will be like
tensor([ 1., 2., 6., 7., 8., 9., 10., 3., 4., 5.])
I'd like to calculate a matrix (shape[N,N]) by multiplying 2 torch vectors A(shape[N,1]) and B=A'(shape[1,N]).
When I use torch.matmul or torch.mm, I got errors or A'A(shape[1,1]).
If A is denoted as
A = [a_1, a_2, ..., a_N]', I want to calculate a matrix C whose (i,j)element is
for i in range(N):
for j in range(N):
C(i,j) = a_i * a_j
I'd like to calculate this quickly. Do you have any ideas?
thank you for your help!
If I understood you correctly you can do something like this :
A = torch.randint(0,5,(5,))
C = (A.view(1, -1) * A.view(-1, 1)).to(torch.float)
it produces :
tensor([[ 1., 4., 3., 3., 0.],
[ 4., 16., 12., 12., 0.],
[ 3., 12., 9., 9., 0.],
[ 3., 12., 9., 9., 0.],
[ 0., 0., 0., 0., 0.]])
which is equivalent to writting :
D = torch.zeros((5,5))
for i in range(5):
for j in range(5):
D[i][j] = A[i] * A[j]
which results in :
tensor([[ 1., 4., 3., 3., 0.],
[ 4., 16., 12., 12., 0.],
[ 3., 12., 9., 9., 0.],
[ 3., 12., 9., 9., 0.],
[ 0., 0., 0., 0., 0.]])
You could simply do the following:
import torch
A = torch.randint(0, 5, (3, 2))
B = torch.randint(0, 5, (2, 3))
A:
tensor([[1, 3],
[2, 1],
[1, 3]])
B:
tensor([[1, 0, 3],
[3, 4, 1]])
C = A # B # python 3.5+
C:
tensor([[10, 12, 6],
[ 5, 4, 7],
[10, 12, 6]])
I have a tensor a with three dimensions. The first dimension corresponds to minibatch size, the second to the sequence length, and the third to the feature dimension. E.g.,
>>> a = torch.arange(1, 13, dtype=torch.float).view(2,2,3) # Consider the values of a to be random
>>> a
tensor([[[ 1., 2., 3.],
[ 4., 5., 6.]],
[[ 7., 8., 9.],
[10., 11., 12.]]])
I have a second, two-dimensional tensor. Its first dimension corresponds to the minibatch size and its second dimension to the sequence length. It contains values in the range of the indices of the third dimension of a. as third dimension has size 3, so b can contain values 0, 1 or 2. E.g.,
>>> b = torch.LongTensor([[0, 2],[1,0]])
>>> b
tensor([[0, 2],
[1, 0]])
I want to obtain a tensor c that has the shape of b and contains all the values of a that are referenced by b.
In the upper scenario I would like to have:
c = torch.empty(2,2)
c[0,0] = a[0, 0, b[0,0]]
c[1,0] = a[1, 0, b[1,0]]
c[0,1] = a[0, 1, b[0,1]]
c[1,1] = a[1, 1, b[1,1]]
>>> c
tensor([[ 1., 5.],
[ 8., 10.]])
How can I create the tensor c fast? Further, I also want c to be differentiable (be able to use .backprob()). I am not too familiar with pytorch, so I am not sure, if a differentiable version of this exists.
As an alternative, instead of c having the same shape as b I could also use a c with the same shape of a, having only zeros, but at the places referenced by b ones. Then I could multiply a and c to obtain a differentiable tensor.
Like follows:
c = torch.zeros(2,2,3, dtype=torch.float)
c[0,0,b[0,0]] = 1
c[1,0,b[1,0]] = 1
c[0,1,b[0,1]] = 1
c[1,1,b[1,1]] = 1
>>> a*c
tensor([[[ 1., 0., 0.],
[ 0., 5., 0.]],
[[ 0., 8., 0.],
[10., 0., 0.]]])
Lets declare necessary variables first: (notice requires_grad in a's initialization, we will use it to ensure differentiability)
a = torch.arange(1,13,dtype=torch.float32,requires_grad=True).reshape(2,2,3)
b = torch.LongTensor([[0, 2],[1,0]])
Lets reshape a and squash minibatch and sequence dimensions:
temp = a.reshape(-1,3)
so temp now looks like:
tensor([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[10., 11., 12.]], grad_fn=<AsStridedBackward>)
Notice now each value of b can be used in each row of temp to get desired output. Now we do:
c = temp[range(len(temp )),b.view(-1)].view(b.size())
Notice how we index temp, range(len(temp )) to select each row and 1D b i.e b.view(-1) to get corresponding columns. Lastly .view(b.size()) brings this array to the same size as b.
If we print c now:
tensor([[ 1., 6.],
[ 8., 10.]], grad_fn=<ViewBackward>)
The presence of grad_fn=.. shows that c requires gradient i.e. its differentiable.