I have a input tensor that is of size [B, N, 3] and I have a test tensor of size [N, 3] . I want to apply a dot product of the two tensors such that I get [B, N] basically. Is this actually possible?
Yes, it's possible:
a = torch.randn(5, 4, 3)
b = torch.randn(4, 3)
c = torch.einsum('ijk,jk->ij', a, b) # torch.Size([5, 4])
Another alternative:
a = torch.randn(5, 4, 3)
b = torch.randn(4, 3)
c = (a * b[None, ...]).sum(dim=-1) # torch.Size([5, 4])
Related
Supposing I have two square matrices A, B of the same size
A = torch.tensor([[1, 2], [3, 4]])
B = torch.tensor([[1, 1], [1, 1]])
And I want a resulting tensor that consists of the row-wise dot product, say
tensor([3, 7]) # i.e. (1*1 + 2*1, 3*1 + 4*1)
What is an efficient means of achieving this in PyTorch?
As you said you can use torch.bmm but you first need to broadcast your inputs:
>>> torch.bmm(A[..., None, :], B[..., None])
tensor([[[3]],
[[7]]])
Alternatively you can use torch.einsum:
>>> torch.einsum('ij,ij->i', A, B)
tensor([3, 7])
import torch
import numpy as np
def row_wise_product(A, B):
num_rows, num_cols = A.shape[0], A.shape[1]
prod = torch.bmm(A.view(num_rows, 1, num_cols), B.view(num_rows, num_cols, 1))
return prod
A = torch.tensor(np.array([[1, 2], [3, 4]]))
B = torch.tensor(np.array([[1, 1], [1, 1]]))
C = row_wise_product(A, B)
Given a matrix B with shape (M, N), where M > N. How to find a vector v (with shape of M) that is perpendicular to all columns in B.
I tried using Numpy numpy.linalg.lstsq method to solve : Bx = 0. 0 here is a vector with M zeros.
It returns a vector of zeros with (N,) shape.
You can use sympy library, like
from sympy import Matrix
B = [[2, 3, 5], [-4, 2, 3], [0, 0, 0]]
V = A.nullspace()[0]
or to find whole nullspace
N = A.nullspace()
Here is what worked for me in case someone else need the answer:
u,s,vh=np.linalg.svd(B)
v=vh[-1:1]
Suppose I have four one-dimensional numpy arrays A, B, C, D, and I want to create a matrix M such that each entry M[i, j, k, l] of the matrix is the tuple (a, b, c, d)
where a = A[i], b = B[j], c = C[k] and d = D[d].
How can I go about constructing it efficiently without loops?
You can create an empty array M with the correct shape (note the 4 in the last dimension -- that's your tuple), and use broadcasting to assign entire rows/columns in M afterwards.
M = np.empty((
len(a), len(b), len(c), len(d), 4
))
M[..., 0] = a[:, None, None, None]
M[..., 1] = b[None, :, None, None]
M[..., 2] = c[None, None, :, None]
M[..., 3] = d[None, None, None, :]
If I have a tensor A which has shape [M, N],
I want to repeat the tensor K times so that the result B has shape [M, K, N]
and each slice B[:, k, :] should has the same data as A.
Which is the best practice without a for loop.
K might be in other dimension.
torch.repeat_interleave() and tensor.repeat() does not seem to work. Or I am using it in a wrong way.
tensor.repeat should suit your needs but you need to insert a unitary dimension first. For this we could use either tensor.unsqueeze or tensor.reshape. Since unsqueeze is specifically defined to insert a unitary dimension we will use that.
B = A.unsqueeze(1).repeat(1, K, 1)
Code Description A.unsqueeze(1) turns A from an [M, N] to [M, 1, N] and .repeat(1, K, 1) repeats the tensor K times along the second dimension.
Einops provides repeat function
import einops
einops.repeat(x, 'm n -> m k n', k=K)
repeat can add arbitrary number of axes in any order and reshuffle existing axes at the same time.
Adding to the answer provided by #Alleo. You can use following Einops function.
einops.repeat(example_tensor, 'b h w -> (repeat b) h w', repeat=b)
Where b is the number of times you want your tensor to be repeated and h, w the additional dimensions to the tensor.
Example -
example_tensor.shape -> torch.Size([1, 40, 50])
repeated_tensor = einops.repeat(example_tensor, 'b h w -> (repeat b) h w', repeat=8)
repeated_tensor.shape -> torch.Size([8, 40, 50])
More examples here - https://einops.rocks/api/repeat/
Repeated values are memory heavy, in most cases best practice is to use broadcasting. So you would use A[:, None, :] which would make A.shape==(M, 1, N).
One case where I would agree to repeating the values, is that of in place operations in the following steps.
As numpy and torch differ in their implementations I like to agnostic (A * torch.ones(K, 1, 1))) followed by a transpose.
tensor.expand might be a better choice than tensor.repeat because according to this: "Expanding a tensor does not allocate new memory, but only creates a new view on the existing tensor where a dimension of size one is expanded to a larger size by setting the stride to 0."
However, be aware that: "More than one element of an expanded tensor may refer to a single memory location. As a result, in-place operations (especially ones that are vectorized) may result in incorrect behavior. If you need to write to the tensors, please clone them first."
M = N = K = 3
A = torch.arange(0, M * N).reshape((M, N))
B = A.unsqueeze(1).expand(M, K, N)
B
'''
tensor([[[0, 1, 2],
[0, 1, 2],
[0, 1, 2]],
[[3, 4, 5],
[3, 4, 5],
[3, 4, 5]],
[[6, 7, 8],
[6, 7, 8],
[6, 7, 8]]])
'''
I have a theano tensor and I would like to clip its values, but each index to a different range.
For example, if I have a vector [a,b,c] , I want to clip a to [0,1] , clip b to [2,3] and c to [3,5].
How can I do that efficiently?
Thanks!
The theano.tensor.clip operation supports symbolic minimum and maximum values so you can pass three tensors, all of the same shape, and it will perform an element-wise clip of the first with respect to the second (minimum) and third (maximum).
This code shows two variations on this theme. v1 requires the minimum and maximum values to be passed as separate vectors while v2 allows the minimum and maximum values to be passed more like a list of pairs, represented as a two column matrix.
import theano
import theano.tensor as tt
def v1():
x = tt.vector()
min_x = tt.vector()
max_x = tt.vector()
y = tt.clip(x, min_x, max_x)
f = theano.function([x, min_x, max_x], outputs=y)
print f([2, 1, 4], [0, 2, 3], [1, 3, 5])
def v2():
x = tt.vector()
min_max = tt.matrix()
y = tt.clip(x, min_max[:, 0], min_max[:, 1])
f = theano.function([x, min_max], outputs=y)
print f([2, 1, 4], [[0, 1], [2, 3], [3, 5]])
def main():
v1()
v2()
main()