Let's say I have two tensors A of shape : [32 , 512] and B of shape : [32 , 512],
And I want to do element-wise multiplication between the vectors of the matrix to get a new matrix of shape : [32 , 1] (first row of A with first row of B, and second row of A with second row of B and so on.. ) , current methods I have tried simply mul the matrix values and not rows and gives a matrix of shape : [32 , 512].
Thanks!
I guess this is equivalent of what you're looking for, first a matrix multiplication, then sum over each row. [associative law]
import torch
a = torch.zeros(31,512)
b = torch.zeros(31,512)
c = torch.sum(torch.mm(a,torch.t(b)), dim=1).unsqueeze(-1)
print(c.shape)
Out:
torch.Size([31, 1])
Related
I have a use-case where I have to do FFT for a given tensor as. Here, FFT is applied to each of the 10 rows, in a column-wise manner which gives the dimension (10, 11) post FFT.
# Random data-
x = torch.rand((10, 20))
# Compute RFFT of 'x'-
x_fft = torch.fft.rfft(x)
# Sanity check-
x.shape, x_fft.shape
# (torch.Size([10, 20]), torch.Size([10, 11]))
# FFT for the first 2 rows are-
x_fft[:2, :]
'''
tensor([[12.2561+0.0000j, 0.7551-1.2075j, 1.1119-0.0458j, -0.2814-1.5266j,
1.4083-0.7302j, 0.6648+0.3311j, 0.3969+0.0632j, -0.8031-0.1904j,
-0.4206+0.9066j, -0.2149+0.9160j, 0.4800+0.0000j],
[ 9.8967+0.0000j, -0.5100-0.2377j, -0.6344+2.2406j, 0.4584-1.0705j,
0.2235+0.4788j, -0.3923+0.8205j, -1.0372-0.0292j, -1.6368+0.5517j,
1.5093+0.0419j, 0.5755-1.2133j, 2.9269+0.0000j]])
'''
# The goal is to have for each row, 1-D vector (of size = 11) as follows:
# So, for first row, the desired 1-D vector (size = 11) is-
[12.2561, 0.0000, 0.7551, -1.2075, 1.1119, -0.0458, -0.2814, -1.5266,
1.4083, -0.7302, 0.6648, 0.3311, 0.3969, 0.0632, -0.8031, -0.1904,
-0.4206, 0.9066, -0.2149, 0.9160, 0.4800, 0.0000]
'''
Here, you are taking the real and imaginary components and placing them adjacent to each other.
Adjacent means:
[a_1_real, a_1_imag, a_2_real, a_2_imag, a_3_real, a_3_imag, ....., a_n_real, a_n_imag]
Since for each row, you get 11 FFT complex numbers, a_n = a_11.
How to go about it?
Your question seems to come down to: how to interleave two tensors together. Given x and y the two tensors. You can do so with a combination of transpose and reshape.
>>> torch.stack((x,y),1).transpose(1,2).reshape(2,-1)
tensor([[ 1.1547e+01, 0.0000e+00, 1.3786e+00, -8.1970e-01, -3.2118e-02,
-2.3900e-02, -3.2898e-01, -3.4610e-01, -1.7916e-01, 1.2308e+00,
-5.4203e-01, 1.2580e-01, 8.5273e-01, 8.9980e-01, -2.7096e+00,
-3.8060e-01, 3.0016e-01, -4.5240e-01, -7.7809e-02, 4.5630e-01,
-4.5805e-03, 0.0000e+00],
[ 1.1106e+01, 0.0000e+00, 1.3362e-01, 1.3830e-01, -7.4233e-01,
7.7570e-01, -9.9461e-01, 1.0834e+00, 1.6952e+00, 5.2920e-01,
-1.1884e+00, -2.5970e-01, -8.7958e-01, 4.3180e-01, -9.3039e-01,
8.8130e-01, -1.0048e+00, 1.2823e+00, 2.0595e-01, -6.5170e-01,
1.7209e+00, 0.0000e+00]])
Given the following tensors x and y with shapes [3,2,3] and [3,2]. I want to multiply the tensors along the 2nd dimension, this is expected to be a kind of dot product and scaling along the axis and return a [3,2,3] tensor.
import torch
a = [[[0.2,0.3,0.5],[-0.5,0.02,1.0]],[[0.01,0.13,0.06],[0.35,0.12,0.0]], [[1.0,-0.3,1.0],[1.0,0.02, 0.03]] ]
b = [[1,2],[1,3],[0,2]]
x = torch.FloatTensor(a) # shape [3,2,3]
y = torch.FloatTensor(b) # shape [3,2]
The expected output :
Expected output shape should be [3,2,3]
#output = [[[0.2,0.3,0.5],[-1.0,0.04,2.0]],[[0.01,0.13,0.06],[1.05,0.36,0.0]], [[0.0,0.0,0.0],[2.0,0.04, 0.06]] ]
I have tried the two below but none of them is giving the desired output and output shape.
torch.matmul(x,y)
torch.matmul(x,y.unsqueeze(1).shape)
What is the best way to fix this?
This is just broadcasted multiply. So you can insert a unitary dimension on the end of y to make it a [3,2,1] tensor and then multiply by x. There are multiple ways to insert unitary dimensions.
# all equivalent
x * y.unsqueeze(2)
x * y[..., None]
x * y[:, :, None]
x * y.reshape(3, 2, 1)
You could also use torch.einsum.
torch.einsum('abc,ab->abc', x, y)
I have the following code segment to generate random samples. The generated samples is a list, where each entry of the list is a tensor. Each tensor has two elements. I would like to extract the first element from all tensors in the list; and extract the second element from all tensors in the list as well. How to perform this kind of tensor slice operation
import torch
import pyro.distributions as dist
num_samples = 250
# note that both covariance matrices are diagonal
mu1 = torch.tensor([0., 5.])
sig1 = torch.tensor([[2., 0.], [0., 3.]])
dist1 = dist.MultivariateNormal(mu1, sig1)
samples1 = [pyro.sample('samples1', dist1) for _ in range(num_samples)]
samples1
I'd recommend torch.cat with a list comprehension:
col1 = torch.cat([t[0] for t in samples1])
col2 = torch.cat([t[1] for t in samples1])
Docs for torch.cat: https://pytorch.org/docs/stable/generated/torch.cat.html
ALTERNATIVELY
You could turn your list of 1D tensors into a single big 2D tensor using torch.stack, then do a normal slice:
samples1_t = torch.stack(samples1)
col1 = samples1_t[:, 0] # : means all rows
col2 = samples1_t[:, 1]
Docs for torch.stack: https://pytorch.org/docs/stable/generated/torch.stack.html
I should mention PyTorch tensors come with unpacking out of the box, this means you can unpack the first axis into multiple variables without additional considerations. Here torch.stack will output a tensor of shape (rows, cols), we just need to transpose it to (cols, rows) and unpack:
>>> c1, c2 = torch.stack(samples1).T
So you get c1 and c2 shaped (rows,):
>>> c1
tensor([0.6433, 0.4667, 0.6811, 0.2006, 0.6623, 0.7033])
>>> c2
tensor([0.2963, 0.2335, 0.6803, 0.1575, 0.9420, 0.6963])
Other answers that suggest .stack() or .cat() are perfectly fine from PyTorch perspective.
However, since the context of the question involves pyro, may I add the following:
Since you are doing IID samples
[pyro.sample('samples1', dist1) for _ in range(num_samples)]
A better way to do it with pyro is
dist1 = dist.MultivariateNormal(mu1, sig1).expand([num_samples])
This tells pyro that the distribution is batched with a batch size of num_samples. Sampling from this will produce
>> dist1.sample()
tensor([[-0.8712, 6.6087],
[ 1.6076, -0.2939],
[ 1.4526, 6.1777],
...
[-0.0168, 7.5085],
[-1.6382, 2.1878]])
Now its easy to solve your original question. Just slice it like
samples = dist1.sample()
samples[:, 0] # all first elements
samples[:, 1] # all second elements
This is for Python 3.x or more specifically Scipy.sparse. I wish to write
import numpy as np, scipy.sparse as sp
A = sp.csc_matrix((4,3))
B = sp.csc_matrix(np.random.rand(2,1))
A[2:-1,0] = B
The last line does not work. I intend the last line to insert the matrix B into A as the block with row 3 to row 4 and column 0. What is the correct way to achieve this assignment without a loop?
The setup
In [219]: from scipy import sparse
In [220]: A = sparse.csr_matrix((4,3))
In [221]: A
Out[221]:
<4x3 sparse matrix of type '<class 'numpy.float64'>'
with 0 stored elements in Compressed Sparse Row format>
In [222]: B = sparse.csr_matrix(np.random.rand(2,1))
In [223]: B
Out[223]:
<2x1 sparse matrix of type '<class 'numpy.float64'>'
with 2 stored elements in Compressed Sparse Row format>
In [224]: B.A
Out[224]:
array([[0.04427272],
[0.03421125]])
Your attempt, WITH ERROR
In [225]: A[2:-1, 0] = B
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-225-0dcce4b72635> in <module>
----> 1 A[2:-1, 0] = B
/usr/local/lib/python3.6/dist-packages/scipy/sparse/_index.py in __setitem__(self, key, x)
111 if not ((broadcast_row or x.shape[0] == i.shape[0]) and
112 (broadcast_col or x.shape[1] == i.shape[1])):
--> 113 raise ValueError('shape mismatch in assignment')
114 if x.size == 0:
115 return
ValueError: shape mismatch in assignment
So let's focus on shapes:
In [226]: A[2:-1, 0].shape
Out[226]: (1, 1)
In [227]: B.shape
Out[227]: (2, 1)
Well duh! we can't put a (2,1) into a (1,1) slot, even with dense arrays.
If we drop the -1, we get a 2 element slot:
In [230]: A.A[2:,0].shape
Out[230]: (2,)
Now the assignment works - with a warning.
In [231]: A[2:, 0] = B
/usr/local/lib/python3.6/dist-packages/scipy/sparse/_index.py:118: SparseEfficiencyWarning: Changing the sparsity structure of a csr_matrix is expensive. lil_matrix is more efficient.
self._set_arrayXarray_sparse(i, j, x)
In [232]: A
Out[232]:
<4x3 sparse matrix of type '<class 'numpy.float64'>'
with 2 stored elements in Compressed Sparse Row format>
In [233]: A.A
Out[233]:
array([[0. , 0. , 0. ],
[0. , 0. , 0. ],
[0.04427272, 0. , 0. ],
[0.03421125, 0. , 0. ]])
As a general rule, we don't make sparse matrices by assigning values to an existing 'empty' one. We make a csr matrix with the coo styles of inputs - the row, col, data arrays.
I have a 2-d placeholder tensor with dimensions of (2,2). How can I expand the columns (same number dimensions) so that the new tensor is (2,3) and assign a constant value to the new column?
For example, the current data may look like
[[2,2], [2,2]]
And I want to transform through tensorflow to (prepending a constant of 1):
[[1,2,2], [1,2,2]]
You can use the tf.concat() op to concatenate a constant with your placeholder:
placeholder = tf.placeholder(tf.int32, shape=[2, 2])
prefix_column = tf.constant([[1], [1]])
expanded_placeholder = tf.concat([prefix_column, placeholder], axis=1)