Why doesn't PyTorch implicitly convert numbers and lists to tensors, when used with torch functions? For example
torch.exp([0, 1, 2])
errors out with
TypeError: exp(): argument 'input' (position 1) must be Tensor, not list
Is it the overheads associated with creating a tensor, which Pytorch doesn't want to implicitly assume?
Related
i want to convert a tensor to numpy
the result of a layer in my code is tensor so i can't convert it to numpy
is there any way to make the result of this layer be EagerTensor
I've been trying to convert a tensor of type:
tensorflow.python.framework.ops.Tensor
to an eagertensor:
<class 'tensorflow.python.framework.ops.EagerTensor'>
Probleme:
'Tensor' object has no attribute 'numpy'
I am working with TensorFlow in Python.
I read through the documentation of
tf.random.truncated_normal
that the input 'shape' gets 1-D tensor or python array, i.e. a vector (according to https://www.tensorflow.org/guide/tensors).
However, with the example I'm using, 'shape' is a 4-D tensor. Or is it considered a vector? Perhaps I have problem with the definition of vectors and tensors?
def weight_variable(shape, name = 'noname'):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial, name = name)
W_conv1 = weight_variable([5, 5, 3, 32], 'W_conv1')
So there is a small mistake you are making in your understanding of a Tensor. A Tensor can have different "ranks". A single scalar such as 1 is a Rank 0 Tensor. A list/vector such as [1,2,3,4] is a Rank 1 Tensor. a 2-D Matrix such as [[0,0],[0,0]] is a Rank 2 Tensor and 3D Matrix are Rank 3 Tensors and so on. So the input you have here is a vector or Rank 1 Tensor not a 4-D Tensor.
Here is a nice blog post about this.
Why does the following error popup? What should be in this range and why? What does -2 dimension mean?
RuntimeError: dimension out of range (expected to be in range of [-2, 1], but got 2)
This code will produce the error
import torch
torch.bmm(torch.randn(1000, 784) , torch.randn(784, 10))
torch.mm:
Performs a matrix multiplication of the matrices mat1 and mat2.
If mat1 is a (n×m) tensor, mat2 is a (m×p) tensor, out will be a (n×p)
tensor.
torch.bmm:
Performs a batch matrix-matrix product of matrices stored in batch1
and batch2.
batch1 and batch2 must be 3-D tensors each containing the same number
of matrices.
If batch1 is a (b×n×m) tensor, batch2 is a (b×m×p) tensor, out will be
a (b×n×p) tensor.
The following code snippet works.
import torch
x = torch.mm(torch.randn(100, 78) , torch.randn(78, 10))
bsize = 16
x = torch.bmm(torch.randn(bsize, 100, 78) , torch.randn(bsize, 78, 10))
The method torch.bmm implements the batch matrix-matrix product. For normal matrix-matrix product you need two have two 2D matrices in order to create the product.
With torch.bmm you can create the product even batch-wize but of course then you need to include the batch dimension, thus you need to two input 3 dimensional matrices.
About the how the are dimensions used in torch.bmm:
If batch1 is a (b×n×m) tensor,
batch2 is a (b×m×p) tensor, output will be a (b×n×p) tensor.
https://pytorch.org/docs/master/torch.html#torch.bmm
I have a series of processed audio files I am using as input into a CNN using Keras. Does the Keras 1D Convolutional layer support variable sequence lengths? The Keras documentation makes this unclear.
https://keras.io/layers/convolutional/
At the top of the documentation it mentions you can use (None, 128) for variable-length sequences of 128-dimensional vectors. Yet at the bottom it declares that the input shape must be a
3D tensor with shape: (batch_size, steps, input_dim)
Given the following example how should I input sequences of variable length into the network
Lets say I have two examples (a and b) containing X 1 dimensional vectors of length 100 that I want to feed into the 1DConv layer as input
a.shape = (100, 100)
b.shape = (200, 100)
Can I use an input shape of (2, None, 100)? Do I need to concatenate these tensors into c where
c.shape = (300, 100)
Then reshape it to be something
c_reshape.shape = (3, 100, 100)
Where 3 is the batch size, 100, is the number of steps, and the second 100 is the input size? The documentation on the input vector is not very clear.
Keras supports variable lengths by using None in the respective dimension when defining the model.
Notice that often input_shape refers to the shape without the batch size.
So, the 3D tensor with shape (batch_size, steps, input_dim) suits perfectly a model with input_shape=(steps, input_dim).
All you need to make this model accept variable lengths is use None in the steps dimension:
input_shape=(None, input_dim)
Numpy limitation
Now, there is a numpy limitation about variable lengths. You cannot create a numpy array with a shape that suits variable lengths.
A few solutions are available:
Pad your sequences with dummy values until they all reach the same size so you can put them into a numpy array of shape (batch_size, length, input_dim). Use Masking layers to disconsider the dummy values.
Train with separate numpy arrays of shape (1, length, input_dim), each array having its own length.
Group your images by sizes into smaller arrays.
Be careful with layers that don't support variable sizes
In convolutional models using variable sizes, you can't for instance, use Flatten, the result of the flatten would have a variable size if this were possible. And the following Dense layers would not be able to have a constant number of weights. This is impossible.
So, instead of Flatten, you should start using GlobalMaxPooling1D or GlobalAveragePooling1D layers.
I trying to compile a model in Keras with an input that is a 2D numpy array.
What I need is to take the vector at the nth place of this 2D array and use it as a tensor 1D tensor for one of the layers.
How do I do it?
Using a lambda layer should do it:
extracted_tensor = Lambda(lambda x: x[:,nth_index,:], output_shape=(1,dim_vector))(input)
extracted_tensor = Flatten()(extracted_tensor)
note that in the x tensor (lambda function), you take the batch dimension into account, but you don't in the output_shape parameter.
I hope this helps
Use tf.gather( input_tensor, indices, axis ) to collect indices along the specified axis.