tensor(1.0000, grad_fn=<SumBackward0>)
tensor([1.])
Why are these two tensors not equivalent in pytorch?
I figured it out: when converting to numpy,
assert torch.sum(state_alphas).detach().numpy() == 1, torch.sum(state_alphas).detach().numpy()
AssertionError: 0.9999994
For some reason even though the Tensor is displayed as 1.000... it is not quite equal to 1.
Related
I want a similar function as in https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.circulant.html to create a circulant matrix using PyTorch. I need this as a part of my Deep Learning model (in order to reduce over-parametrization in some of my Fully Connected layers as suggested in https://arxiv.org/abs/1907.08448 (Fig.3))
The input of the function shall be a 1D torch tensor, and the output should be the 2D circulant matrix.
You can make use of unfold to extract sliding windows. But to get the correct order you need to flip (later unflip) the tensors, and first concatenate the flipped tensor to itself.
circ=lambda v:torch.cat([f:=v.flip(0),f[:-1]]).unfold(0,len(v),1).flip(0)
Here is a generic function for pytorch tensors, to get the circulant matrix for one dimension. It's based on unfold and it works for 2d circulant matrix or high-dimension tensors.
def circulant(tensor, dim):
"""get a circulant version of the tensor along the {dim} dimension.
The additional axis is appended as the last dimension.
E.g. tensor=[0,1,2], dim=0 --> [[0,1,2],[2,0,1],[1,2,0]]"""
S = tensor.shape[dim]
tmp = torch.cat([tensor.flip((dim,)), torch.narrow(tensor.flip((dim,)), dim=dim, start=0, length=S-1)], dim=dim)
return tmp.unfold(dim, S, 1).flip((-1,))
Essentially, this is a PyTorch version of scipy.linalg.circulant and works for multi-dimension tensors.
Also a similar question: Create array/tensor of cycle shifted arrays
The following screenshots show that torch.mv is unusable in a situation that obviously seem to be correct... how is this possible, any idea what can be the problem?
this first image shows the correct situation, where the vector has 10 rows for a matrix of 10 columns, but I showed the other also just in case. Also swapping w.mv(x) for x.mv(w) does not make a difference.
However, the # operator works... the thing is that for my own reasons I want to use mv, so I would like to know what the problem is.
According to documentation:
torch.mv(input, vec, *, out=None) → Tensor
If input is a (n×m) tensor, vec is a 1-D tensor of size m, out will be 1-D of size n.
The x here should be 1-D, but in your case it's 10x1 (2D). You can remove extra dimension (or create a single dimension x)
>>> w.mv(x.squeeze())
tensor([ 0.1432, -2.0639, -2.1871, -1.8837, 0.7333, -0.4000, 0.4023, -1.1318,
0.0423, -1.2136])
>>> w # x
tensor([[ 0.1432],
[-2.0639],
[-2.1871],
[-1.8837],
[ 0.7333],
[-0.4000],
[ 0.4023],
[-1.1318],
[ 0.0423],
[-1.2136]])
What is the difference between Tensor.size and Tensor.shape in Pytorch?
I want to get the number of elements and the dimensions of Tensor. For example for a tensor with the dimensions of 2 by 3 by 4 I expect 24 for number of elements and (2,3,4) for dimension.
Thanks.
.shape is an alias for .size(), and was added to more closely match numpy, see this discussion here.
.shape is an attribute of the tensor whereas size() is a function. They both return the same value.
I get the same results when using either the python sum or torch.sum so why did torch implement a sum function? Is there a difference between them?
nothing, torch.sum calls tensor.sum and python's sum calls __add__ (or __radd__ when needed) which calls tensor.sum again
so the only difference is in the number of function calls, and tensor.sum() should be the fastest (when you have small tensors and the function call's overhead is considerable)
It appears python's sum can take generators as input, whereas torch.sum cannot:
import torch
print( sum( torch.ones(1)*k for k in torch.arange(10)))
returns tensor([45.]), whereas:
print( torch.sum( torch.ones(1)*k for k in torch.arange(10)))
raises TypeError: sum(): argument 'input' (position 1) must be Tensor, not generator
I'm assuming that pyTorch's backpropagation would get in trouble with lazy evaluation of the generator, but not sure about that, yet.
I am using a DBN classification code. I have one output value which is an integer(value of the output integer can be anything 110,12 etc). So basically I am trying to predict something using the classification code by setting its no_of_outputs=1. But I think no_of_output=1 is messing up with the code by giving the error:
ValueError: y_i value out of bounds
Apply node that caused the error: CrossentropySoftmaxArgmax1HotWithBias(_dot22.0, b, Elemwise{Cast{int32}}.0)
Inputs shapes: [(10, 2), (2,), (10,)]
Inputs strides: [(16, 8), (8,), (4,)]
Inputs types: [TensorType(float64, matrix), TensorType(float64, vector), TensorType(int32, vector)]
Use the Theano flag 'exception_verbosity=high' for a debugprint of this apply node.
Kindly help me solve it.
That's index value out of bound error, you should check your codes to make sure:
Your index value is not NaN
If your index value is a valid integer, you should check whether it's out of bound actually
You can print your value by compiling a theano function, which returns the parameters you want to print.