I have one tensor in shape (2, G) and another in shape (N, 2).
I need to add them in such a way that the output is (N, 2, G), meaning that the first tensor is replicated to (N, 2, G) and then the second tensor is added to each matrix along the third dimension. (or vice versa: the second tensor is replicated to (N, 2, G) and the first one is added to every sub-tensor along the first dimension).
How can this done efficiently in Theano?
Thanks.
In an attempt to understand the problem, the following example is assumed to be representative.
If
A = [[1, 2, 3],
[4, 5, 6]]
and
B = [[1, 2],
[3, 4],
[5, 6],
[7, 8]]
then the result should be
C = [[[ 2. 3. 4.]
[ 6. 7. 8.]]
[[ 4. 5. 6.]
[ 8. 9. 10.]]
[[ 6. 7. 8.]
[ 10. 11. 12.]]
[[ 8. 9. 10.]
[ 12. 13. 14.]]]
Here G=3 and N=4.
To achieve this in Theano, one need only add new broadcastable dimensions and rely on broadcasting to get the desired result.
import numpy
import theano
import theano.tensor as tt
x = tt.matrix()
y = tt.matrix()
z = x.dimshuffle('x', 0, 1) + y.dimshuffle(0, 1, 'x')
f = theano.function([x, y], outputs=z)
print f(numpy.array([[1, 2, 3], [4, 5, 6]]), numpy.array([[1, 2], [3, 4], [5, 6], [7, 8]]))
Related
I want to do an operation similar to matrix multiplication, except instead of multiplying I want to check equality. The effect that I want to achieve is similar to the following:
a = torch.Tensor([[1, 2, 3], [4, 5, 6]]).to(torch.uint8)
b = torch.Tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]]).to(torch.uint8)
result = [[sum(a[i] == b [j]) for j in range(len(b))] for i in range(len(a))]
Is there a way that I can use einsum, or any other function in pytorch to achieve the above efficiently?
You can use torch.repeat and torch.repeat_interleave:
a = torch.Tensor([[1, 2, 3], [4, 5, 6]])
b = torch.Tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
mask = a.repeat_interleave(3, dim=0) == b.repeat((2, 1))
torch.sum(mask, axis=1).reshape(a.shape)
# output
tensor([[3, 0, 0],
[0, 3, 0]])
You can make use of the broadcasting to do the same, for instance with
result = (a[:, None, :] == b[None, :, :]).sum(dim=2)
Here None just introduces a dummy dimensions - alternatively you can use the less visual .unsqueeze() instead.
matrix multiplication is ij,jk->ik in einsum notation, all of these operations are equivalent with varying levels of verbosity:
a # b
torch.einsum("ij,jk", a, b)
torch.einsum("ij,jk->ik", a, b)
(a[:,:,None] * b[None,:,:]).sum(1)
"multiply i and k dimensions and reduce j dimension"
i, j, k i, j, k
a: (2, 3) => (2, 3, None)
b: (3, 3) (None, 3, 3)
It should now be clear from this function decomposition that multiplication can be replaced with any binary operation, e.g. the equality operation.
Unfortunately, there is no generalized form of einsum (AFAIK) in pytorch that swaps the multiplication "out-of-the-box". There is however the einops library which is basically a wrapper around deep learning frameworks such as PyTorch.
i need to write a function in python that takes a matrix as an argument and divides each element of the matrix by 2 if the element is an even number (otherwise, does nothing).
i also need to use list comprehension for this.
as an example, if i have a matrix like m = [[5, 4], [2, 3], [6, 7]] output: [[5, 2], [1, 3], [3, 7]]
Thanks.
def f(matrix):
return [ [x//2 if x%2==0 else x for x in m ] for m in matrix]
print(f([[5, 4], [2, 3], [6, 7]]))
I'm trying to get used to the Embedding class in the PyTorch nn module.
I've noticed that quite a few other people have had the same problem as myself, and therefore posted questions on the PyTorch discussion forum and on Stack Overflow, but I'm still having some confusion.
According to the official documentation, the arguments that are passed are num_embeddings and embedding_dim which each refer to how large our dictionary (or vocabulary) is and how many dimensions we want our embeddings to be, respectively.
What I'm confused about is how exactly I should interpret those. For example, the small practice code that I ran:
import torch
import torch.nn as nn
embedding = nn.Embedding(num_embeddings=10, embedding_dim=3)
a = torch.LongTensor([[1, 2, 3, 4], [4, 3, 2, 1]]) # (2, 4)
b = torch.LongTensor([[1, 2, 3], [2, 3, 1], [4, 5, 6], [3, 3, 3], [2, 1, 2],
[6, 7, 8], [2, 5, 2], [3, 5, 8], [2, 3, 6], [8, 9, 6],
[2, 6, 3], [6, 5, 4], [2, 6, 5]]) # (13, 3)
c = torch.LongTensor([[1, 2, 3, 2, 1, 2, 3, 3, 3, 3, 3],
[2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]]) # (2, 11)
When I run a, b, and c through the embedding variable, I get embedded results of shapes (2, 4, 3), (13, 3, 3), (2, 11, 3).
What's confusing me is that I thought of the number of samples we have exceeds the predefined number of embeddings, we should get an error? Since the embedding I've defined has 10 embeddings, shouldn't b give me an error since it is a tensor containing 13 words of dimension 3?
In your case, here is how your input tensor are interpreted:
a = torch.LongTensor([[1, 2, 3, 4], [4, 3, 2, 1]]) # 2 sequences of 4 elements
Moreover, this is how your embedding layer is interpreted:
embedding = nn.Embedding(num_embeddings=10, embedding_dim=3) # 10 distinct elements and each those is going to be embedded in a 3 dimensional space
So, it doesn't matter if your input tensor has more than 10 elements, as long as they are in the range [0, 9]. For example, if we create a tensor of two elements such as:
d = torch.LongTensor([[1, 10]]) # 1 sequence of 2 elements
We would get the following error when we pass this tensor through the embedding layer:
RuntimeError: index out of range: Tried to access index 10 out of table with 9 rows
To summarize num_embeddings is total number of unique elements in the vocabulary, and embedding_dim is the size of each embedded vector once passed through the embedding layer. Therefore, you can have a tensor of 10+ elements, as long as each element in the tensor is in the range [0, 9], because you defined a vocabulary size of 10 elements.
My code generates numpy 1D arrays of integers. Here's an example.
arr = np.array([-8, 7, -5, 2, -7, 8, -6, 3, 5])
There are two steps I need to take with this array, but I'm new enough at Python that I'm at a loss how do this efficiently. The two steps are:
a) Identify the 1st element of pairs having sum == 0. For arr, we have (-8, 7, -5).
b) Now I need to find the difference in indices for each of the pairs identified in a).
The difference in indices for (-8,8) is 5, for (7,-7) is 3,
and for (-5,5) is 6.
Ideally, the output could be a 2D array, something like:
[[-8, 5],
[ 7, 3],
[-5, 6]]
Thank you for any assistance.
Here is my solution:
arr = np.array([-8, 7, -5, 2, -7, 8, -6, 3, 5])
output = list()
for i in range(len(arr)):
for j in range(len(arr)-i):
if arr[i] + arr[i+j] == 0:
output.append([arr[i],j])
print(output)
[[-8, 5], [7, 3], [-5, 6]]
I have two comments also:
1) You can transfer the list to the numpy array by np.asarray(output)
2) Imagine you have list [8, -8, -8]. If you want to calculate distance of the first pair only, you can simply add break after the appending procedure.
How does numpy's matrix class work? I understand it will likely be removed in the future, so I am trying to understand how it works, so I can do the same with ndarrrays.
>>> x=np.matrix([[1,1,1],[2,2,2],[3,3,3]])
>>> x[:,0] + x[0,:]
matrix([[2, 2, 2],
[3, 3, 3],
[4, 4, 4]])
Seems like a row of ones got added to every row.
>>> x=np.matrix([[1,2,3],[1,2,3],[1,2,3]])
>>> x[0,:] + x[:,0]
matrix([[2, 3, 4],
[2, 3, 4],
[2, 3, 4]])
Now it seems like a column of ones got added to every column. What it does it with the identity is even weirder,
>>> x=np.matrix([[1,0,0],[0,1,0],[0,0,1]])
>>> x[0,:] + x[:,0]
matrix([[2, 1, 1],
[1, 0, 0],
[1, 0, 0]])
EDIT:
It seems if you take a (N,1) shape matrix and add it to a (N,1) shape matrix, then of one these is replicated to form a (N,N) matrix and the other is added to every row or column of this new matrix. It seems to be a convenience restricted to vectors of the right sizes. A nice use case was networkx's implementation of Floyd-Warshal.
Is there an equivalently convenient one-liner for this using standard numpy ndarrays?