my pytorch code:
import torch
x = torch.tensor([[0.3992, 0.2908, 0.9004, 0.4850, 0.6004],
[0.5735, 0.9006, 0.6797, 0.4152, 0.1732]])
print(x.shape)
mask = torch.tensor([[False, False, True, False, True],
[ True, True, True, False, False]])
print(mask.shape)
y = torch.tensor([[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.]])
print(y.shape)
y.masked_scatter_(mask, x)
print(y)
result is:
torch.Size([2, 5])
torch.Size([2, 5])
torch.Size([2, 5])
tensor([[0.0000, 0.0000, 0.3992, 0.0000, 0.2908],
[0.9004, 0.4850, 0.6004, 0.0000, 0.0000]])
i think the result answer is:
tensor([[0.0000, 0.0000, 0.9004, 0.0000, 0.6004],
[0.5375, 0.9006, 0.6797, 0.0000, 0.0000]])
my pytorch version is pytorch1.4
You are right, this is confusing and there is virtually no documentation.
However, the way scatter works (as you have discovered) is that the ith True in a row is given the ith value from the source. So not the value corresponding to the position of the True.
Luckily what you are trying to do can easily be achieved using the normal indexing notation:
>>> y[mask] = x[mask]
>>> y
tensor([[0.0000, 0.0000, 0.9004, 0.0000, 0.6004],
[0.5735, 0.9006, 0.6797, 0.0000, 0.0000]])
Related
From a DGL graph I want to see the adjacency matrix with
adjM = g.adjacency_matrix()
adjM
and I get the following which is fine:
tensor(indices=tensor([[0, 0, 0, 1],
[1, 2, 3, 3]]),
values=tensor([1., 1., 1., 1.]),
size=(4, 4), nnz=4, layout=torch.sparse_coo)
Now I want to have the adjacency matrix and the node values each by itself. I imagine something of this kind:
adjMatrix = adjM.indices # or
adjMatrix = adjM[0]
nodeValues = adjM.values # or
nodeValues = adjM[1]
But this form is not estimated by pyTorch/DGL.
My beginner's question:
how to do this correctly and sucsessfully? and
is there a tutorial for a nuby? ( I have searched a lot just for this detail...!)
Click here!
You will find the usage of dgl.adj(). As the doc said, the return is an adjacency matrix, and the return type is the SparseTensor.
I noticed that the output that you post is a SparseTensor.
You can try it as follows then you can get the entire adj_matrix
I create a dgl graph g, get the adjacency matrix as adj
g = dgl.graph(([0, 1, 2], [1, 2, 3]))
adj = g.adj()
adj
output is:
tensor(indices=tensor([[0, 1, 2],
[1, 2, 3]]),
values=tensor([1., 1., 1.]),
size=(4, 4), nnz=3, layout=torch.sparse_coo)
We can find that adj is the presence of sparse, and the sparse type is coo, we can use the following code to verify if adj is a SparseTensor
adj.is_sparse
output :
True
so we can use to_dense() get the original adj matrix
adj.to_dense()
the result is:
tensor([[0., 1., 0., 0.],
[0., 0., 1., 0.],
[0., 0., 0., 1.],
[0., 0., 0., 0.]])
When you have a problem with DGL you can check the Deep Graph Library Tutorials and Documentation.
I have the following function which works, but just not for half precision values (get a NotImplemented error for kthvalue).
def suppress_small_probabilities(probabilities: torch.FloatTensor, k: int) -> torch.FloatTensor:
kth_largest, _ = (-probabilities).kthvalue(k, dim=-1, keepdim=True)
return probabilities * (probabilities >= -kth_largest)
How would you do the equivalent without using kthvalue? I'm guessing topk has something to do with it, but I want to suppress the smaller values. probabilities is of size batch_size x 1000.
Implement your own topk, e.g.
def mytopk(xs: Tensor, k: int) -> Tensor:
mask = torch.zeros_like(xs)
batch_idx = torch.arange(0, len(xs))
for _ in range(k):
_, index = torch.where(mask == 0, xs, -1e4).max(-1)
mask[(batch_idx, index)] = 1
return mask
This will return a boolean mask tensor where the row-wise top-k elements will have value 1, rest 0.
Then use the mask to index your original tensor, e.g.
xs = torch.rand(3, 5, dtype=torch.float16)
# tensor([[0.0626, 0.9620, 0.5596, 0.4423, 0.1932],
# [0.5289, 0.0857, 0.7802, 0.7730, 0.4807],
# [0.8272, 0.5016, 0.1169, 0.4372, 0.1843]], dtype=torch.float16)
mask = mytopk(xs, 2)
# tensor([[0., 1., 1., 0., 0.],
# [0., 0., 1., 1., 0.],
# [1., 1., 0., 0., 0.]])
top_only = torch.where(mask == 1, xs, 0)
# tensor([[0.0000, 0.9620, 0.5596, 0.0000, 0.0000],
# [0.0000, 0.0000, 0.7802, 0.7730, 0.0000],
# [0.8271, 0.5016, 0.0000, 0.0000, 0.0000]], dtype=torch.float16)
I'm trying to implement an einsum operation as matrix multiplication in pytorch. In the example below,
degree_matrix_hat is a matrix with shape (64, 115) and edge_features_reduced is a tensor of shape (64, 115, 115).
Using torch.einsum I get the following output:
ein1 = torch.einsum("bi,bij->bij",degree_matrix_hat, edge_features_reduced)
ein1[0],ein1.size()
(tensor([[0.7071, 0.7071, 0.0000, ..., 0.0000, 0.0000, 0.0000],
[0.5774, 0.5774, 0.5774, ..., 0.0000, 0.0000, 0.0000],
[0.0000, 0.5774, 0.5774, ..., 0.0000, 0.0000, 0.0000],
...,
[0.0000, 0.0000, 0.0000, ..., 1.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, ..., 0.0000, 1.0000, 0.0000],
[0.0000, 0.0000, 0.0000, ..., 0.0000, 0.0000, 1.0000]]),
torch.Size([64, 115, 115]))
However, I get a different output using using torch.matmul:
matmul1=torch.matmul(degree_matrix_hat, edge_features_reduced)
matmul1[0],matmul1.size()
(tensor([[1.2845, 1.8618, 1.7321, ..., 1.0000, 1.0000, 1.0000],
[1.2845, 1.7845, 1.6547, ..., 1.0000, 1.0000, 1.0000],
[1.2071, 1.7845, 1.6547, ..., 1.0000, 1.0000, 1.0000],
...,
[1.2071, 1.7845, 1.6547, ..., 1.0000, 1.0000, 1.0000],
[1.2845, 1.7845, 1.6547, ..., 1.0000, 1.0000, 1.0000],
[1.2071, 1.7845, 1.6547, ..., 1.0000, 1.0000, 1.0000]]),
torch.Size([64, 64, 115]))
I'm quite new to linear algebra so I probably missed something here. What is the exact expression torch.einsum is solving in this case?
I'm learning the Xarray library and I'm trying to create Xarrays for 3D data. The following snippet does not work:
>>> colx = ['c1', 'c2', 'c3', 'c4']
>>> coly = ['a1', 'a2', 'a3']
>>> colz = ['b1', 'b2']
>>> t = xr.DataArray(np.zeros((4,3,2)),dims = {'x', 'y','z'}, coords={'x':colx, 'y': coly, 'z': colz})
>>> t
ValueError: conflicting sizes for dimension 'x': length 3 on the data but length 4 on coordinate 'x'
The following code works, but I need to permute round the lenghts of the x,y,z dimensions to create the Xarray. The dimensions (4,3,2) get permuted to (3,2,4).
>>> colx = ['c1', 'c2', 'c3', 'c4']
>>> coly = ['a1', 'a2', 'a3']
>>> colz = ['b1', 'b2']
>>> t = xr.DataArray(np.zeros((4,3,2)),dims = {'x', 'y','z'}, coords={'x':coly, 'y': colz, 'z': colx})
>>> t
<xarray.DataArray (z: 4, x: 3, y: 2)>
array([[[0., 0.],
[0., 0.],
[0., 0.]],
[[0., 0.],
[0., 0.],
[0., 0.]],
[[0., 0.],
[0., 0.],
[0., 0.]],
[[0., 0.],
[0., 0.],
[0., 0.]]])
Coordinates:
* x (x) <U2 'a1' 'a2' 'a3'
* y (y) <U2 'b1' 'b2'
* z (z) <U2 'c1' 'c2' 'c3' 'c4'
Is this normal?
Why is this happening?
How do the permutations work for higher-dimensional data?
Specifying dims as a list, e.g. dims=['x', 'y', 'z'], instead of a set does the trick, see comment by spencerclark.
I'm looking for a way to expand the size of an image by adding 0 values to the right & lower edges of it. My initial plan is to use nn.padding to add the edge, until I encounter this error:
File "/home/shared/virtualenv/dl-torch/lib/python3.7/site-packages/torch/nn/functional.py", line 2796, in pad
assert len(pad) % 2 == 0, 'Padding length must be divisible by 2'
AssertionError: Padding length must be divisible by 2
It appears that torch tries to pad the image from both side! Is there an easy way to override this and fill the tensor into the upper-left side of another image?
the only way I know is:
with torch.no_grad(): # assuming it's for init
val = torch.distributions.MultivariateNormal(loc=zeros(2), scale=torch.eye(2))
w.data = val
but I doubt it's recommended.
Answering the title of the question.
With nn.ConstantPad2d, you can specify the number of padding elements in all four directions separately.
>>> t = torch.randn(2,3)
>>> t
tensor([[ 0.1254, 0.6358, 0.3243],
[ 0.7005, -0.4931, 1.0582]])
>>> p = torch.nn.ConstantPad2d((0, 4, 0, 2), 0)
>>> p(t)
tensor([[ 0.1254, 0.6358, 0.3243, 0.0000, 0.0000, 0.0000, 0.0000],
[ 0.7005, -0.4931, 1.0582, 0.0000, 0.0000, 0.0000, 0.0000],
[ 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[ 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000]])
I had a similar problem and wanted to initialize a image tensor with a specific color. I solved it as follows:
Let X be a tensor of shape (h, w, dim) and let dim hold 3 values (r,g,b).
If you want to initialize your tensor X with the rgb color 226, 169, 41 you could do something like:
index_0 = torch.tensor([0]) # 226
index_1 = torch.tensor([1]) #169
index_2 = torch.tensor([2]) #41
X.index_fill_(2, index_0, 226)
X.index_fill_(2, index_1, 169)
X.index_fill_(2, index_2, 41)