I have a PyTorch tensor as (2,3,4) dimension I want to make this tensor like (3,2,4) dimension. But I want a value of (:,0,:) equal to (0,:,:). I tried torch.reshape and torch.view but I cannot get the expected result.
How can I do it in PyTorch?
Try this:
import torch
x = torch.randn(2, 0, 4)
print(x.shape)
x = x.permute(1, 0, 2)
print(x.shape)
I'm a beginner in pytorch and I'm probably stuck on a relatively trivial problem, but it's not clearing up for me at the moment.
When calculating the gradient of a tensor I get a constant gradient of 1.
Shouldn't the gradient of a constant however result in 0?
Here is a minimal example:
import torch
x = torch.tensor(50., requires_grad=True)
y = x
y.backward()
print(x.grad)
#Ouput: tensor(1.)
So why is the ouput 1 and not 0?
You are not computing the gradient of a constant, but that of the variable x which has a constant value 50. The derivative of x with respect to x is 1.
When I do
import torch
x = torch.tensor(1)
print(x.dim())
it gives 0. I would like to create an n-dimensional scalar tensor. The only way I can do this is
torch.tensor(1).reshape([1,1,1,1,1,1,1])
but I'm wondering if there's an way to do it from just the torch.tensor construction?
I am trying to multiply each layer of a tensor with the first layer of the tensor.
x1 = bert_model_1([x1_in, x2_in])
x1_begin = Lambda(lambda x: x[:,0])(x1) #obtain the first layer of the bert tensor
x1_begin = Lambda(keras.layers.multiply(x11, x1_begin) for x11 in x1)([x1, x1_begin])
when i ran the code above, i keep getting the following errors,
<generator object build_corrector. . at 0x00000249DB234C48> is not a callable object.
the error seems to happen in the last line, how do i iterate each layer in the tensor?
I saw a sudoku solver CNN uses a sparse categorical cross-entropy as a loss function using the TensorFlow framework, I am wondering if there is a similar function for Pytorch? if not could how could I potentially calculate the loss of a 2d array using Pytorch?
Here is an example of usage of nn.CrossEntropyLoss for image segmentation with a batch of size 1, width 2, height 2 and 3 classes.
Image segmentation is a classification problem at pixel level. Of course you can also use nn.CrossEntropyLoss for basic image classification as well.
The sudoku problem in the question can be seen as an image segmentation problem where you have 10 classes (the 10 digits) (though Neural Networks are not appropriate to solve combinatorial problems like Sudoku which already have efficient exact resolution algorithms).
nn.CrossEntropyLoss accepts ground truth labels directly as integers in [0, N_CLASSES[ (no need to onehot encode the labels):
import torch
from torch import nn
import numpy as np
# logits predicted
x = np.array([[
[[1,0,0],[1,0,0]], # predict class 0 for pixel (0,0) and class 0 for pixel (0,1)
[[0,1,0],[0,0,1]], # predict class 1 for pixel (1,0) and class 2 for pixel (1,1)
]])*5 # multiply by 5 to give bigger losses
print("logits map :")
print(x)
# ground truth labels
y = np.array([[
[0,1], # must predict class 0 for pixel (0,0) and class 1 for pixel (0,1)
[1,2], # must predict class 1 for pixel (1,0) and class 2 for pixel (1,1)
]])
print("\nlabels map :")
print(y)
x=torch.Tensor(x).permute((0,3,1,2)) # shape of preds must be (N, C, H, W) instead of (N, H, W, C)
y=torch.Tensor(y).long() # shape of labels must be (N, H, W) and type must be long integer
losses = nn.CrossEntropyLoss(reduction="none")(x, y) # reduction="none" to get the loss by pixel
print("\nLosses map :")
print(losses)
# notice that the loss is big only for pixel (0,1) where we predicted 0 instead of 1