I'm posting this question for those who made the same error I did. I got this error when trying to compute my gradients:
criterion = torch.nn.CrossEntropyLoss()
loss = criterion(y_hat, y_truth)
loss.backwards()
It's loss.backward(), not loss.backwards().
Related
While trying to script Stable Diffusion model using torch.jit.script(), I got this following error:
AttributeError: 'TimestepEmbedSequential' object has no attribute '__globals__'
I'm trying to export this model to ONNX and found out that running torch.onnx.export() will torch.jit.trace the models, which unrolls every loops, so I'm trying to use script first.
When I follow the traceback, the error occurs in this function while reading fn.__globals__ in _jit_internal.py from torch
def get_closure(fn):
"""
Get a dictionary of closed over variables from a function
"""
captures = {}
captures.update(fn.__globals__)
for index, captured_name in enumerate(fn.__code__.co_freevars):
captures[captured_name] = fn.__closure__[index].cell_contents
return captures
code for scripting is as follows:
stablediffusion_wrapper = StableDiffusionWrapper(model, sampler, opt)
scripted_module = torch.jit.script(stablediffusion_wrapper, example_inputs=[(token_dummy, img_dummy)])
and TimestepEmbedSequential module:
class TimestepEmbedSequential(nn.Sequential, TimestepBlock):
"""
A sequential module that passes timestep embeddings to the children that
support it as an extra input.
"""
def forward(self, x, emb, context=None):
for layer in self:
if isinstance(layer, TimestepBlock):
x = layer(x, emb)
elif isinstance(layer, SpatialTransformer):
x = layer(x, context)
else:
x = layer(x)
return x
Any suggestions how can I figure it out?
I tried to set #torch.jit.export decorator to the parent class TimestepBlock, but showed no effect. Actually, I have no idea what to look for this problem. I would appreciate any suggestions. Thank you
lr=0.001
x=np.linspace(-6,6,120)
y=0.5+3*x-x**2+np.exp(-0.4*x)
x=torch.from_numpy(x)
y=torch.from_numpy(y)
w0=torch.tensor(0.1,requires_grad=True)
w1=torch.tensor(0.1,requires_grad=True)
w2=torch.tensor(0.1,requires_grad=True)
w3=torch.tensor(0.1,requires_grad=True)
Y=w0+w1*x+w2*x**2+torch.exp(w3*x)
#opt=torch.optim.Adam([w0,w1,w2,w3],0.001)
L=nn.MSELoss()
for i in range(20):
opt.zero_grad()
err=L(Y,y)
print(err)
err.backward(retain_graph=True)
with torch.no_grad():
w0=w0-lr*w0.grad
w1=w1-lr*w1.grad
w2=w2-lr*w2.grad
w3=w3-lr*w3.grad
w0.grad.zero_()
w1.grad.zero_()
w2.grad.zero_()
w3.grad.zero_()
from the line w0 = w0 - lr * w0.grad:
===> unsupported operand type(s) for *: 'float' and 'NoneType'
from the line w0.grad.zero_():
===> 'NoneType' object has no attribute 'zero_'
comes up.
How should I fix it?
And
x=np.linspace(-6,6,120)
y=0.5+3*x-x**2+np.exp(-0.4*x)
x=torch.from_numpy(x)
y=torch.from_numpy(y)
w0=torch.tensor(0.1,requires_grad=True)
w1=torch.tensor(0.1,requires_grad=True)
w2=torch.tensor(0.1,requires_grad=True)
w3=torch.tensor(0.1,requires_grad=True)
Y=w0+w1*x+w2*x**2+torch.exp(w3*x)
opt=torch.optim.Adam([w0,w1,w2,w3],0.001)
L=nn.MSELoss()
for i in range(20):
opt.zero_grad()
err=L(Y,y)
print(err)
err.backward(retain_graph=True)
opt.step()
If I try this code, then err is not updated. What is the problem? and how should I fix it
Further, should the input of nn.MSELoss() be torch.double? Sometimes I get expected dtype Double but got dtype Float error.
What should be the type of parameters w0, w1, ...?
You should do the following.
for i in range(20):
Y=w0+w1*x+w2*x**2+torch.exp(w3*x)
err=L(Y,y)
print(err)
err.backward(retain_graph=True)
with torch.no_grad():
w0.add_(-lr*w0.grad)
w1.add_(-lr*w1.grad)
w2.add_(-lr*w2.grad)
w3.add_(-lr*w3.grad)
w0.grad.zero_()
w1.grad.zero_()
w2.grad.zero_()
w3.grad.zero_()
Please note, you are performing Y=w0+w1*x+w2*x**2+torch.exp(w3*x) outside of the for loop which is wrong, and as a result err was not updating.
dtype=torch.float64 should be fine.
When I have trained a model and save the params as hdf5 file, and then I try to evaluate the performance of the model on test_dataset, but there is something wrong in param metrics of model.compile, if I set metrics as
model.compile(optimizer=adam,
loss=losses.sparse_categorical_crossentropy,
metrics=[cus_acc, miou])
It will occurs an Error as follow:
tensorflow.python.framework.errors_impl.InvalidArgumentError: assertion failed: [`labels` out of bound] [Condition x < y did not hold element-wise:] [x (metrics/cus_acc/confusion_matrix/control_dependency:0) = ] [107 118 135...] [y (metrics/cus_acc/confusion_matrix/Cast_2:0) = ] [2] [[Node: metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert = Assert[T=[DT_STRING, DT_STRING, DT_STRING, DT_INT64, DT_STRING, DT_INT64], summarize=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"](metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch/_3827, metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_0, metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_1, metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_2, metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_1/_3829, metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_4, metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_2/_3831)]]
Notes that optimizer adam have been defined, and cus_acc and miou are customized metrics. If delete the metrics it works, so I think there must be something wrong in it, miou and cus_acc are both calculate by the confusion_matrix.
My Question is what makes the error occurs and how to use evaluate_generator to evaluate model performance in keras, if you can provide example code that the best~
Any help would be appreciated. Thanks in advance. :D
It has been solved.
The error occurs when creating confusion_matrix, the nb_classes i.e. 0, 1, 2 must match the label in the ground-truth 0, 1, 2. For example, the ground-truth is not preprocessed and the value of pixels are 0, 127, 255. And then the InvalidArgumentError: assertion failed will occur.
What made my code wrong is not due to the customized metrics, it is when I create a data_generator for ground-truth, the param directory was set the same as the images.
I am training a neural network and a part of my code has returned the following error:
def plot_confusion_matrix(truth,
predictions,
classes,
normalize=False,
save=False,
cmap=plt.cm.Oranges,
path="confusion_matrix.png"):
acc = (np.array(truth) == np.array(predictions))
size = float(acc.shape[0]) #error
acc = np.sum(acc.astype("int32")) / size
(...)
AttributeError: 'bool' object has no attribute 'shape'
function call
pred = pred.numpy()
plot_confusion_matrix(truth=labels.numpy(),
predictions=pred,
save=False,
path="logref_confusion_matrix.png",
classes=["forward", "left", "right"])
Where the thuth represents the labels of Y and predictions the array of prediction, both with shape 32, 3. I checked the update on numpy, ipython etc and all are updated, tried some modification, but without success.
The only reason that acc would be a boolean and not a numpy array of booleans is that you are passing in a singular value for truth and predictions. In the code you provided, there would be no error for an actual array of 32x3. Look at the rest of your code and make sure you actually pass in an array to np.array() instead of singular values.
I am trying to experiment gensim doc2vec, by using following code. As far as I understand from tutorials, it should work. However it gives AttributeError: 'list' object has no attribute 'words'.
from gensim.models.doc2vec import LabeledSentence, Doc2Vec
document = LabeledSentence(words=['some', 'words', 'here'], tags=['SENT_1'])
model = Doc2Vec(document, size = 100, window = 300, min_count = 10, workers=4)
So what did I do wrong? Any help please. Thank you. I am using python 3.5 and gensim 0.12.4
Input to gensim.models.doc2vec should be an iterator over the LabeledSentence (say a list object). Try:
model = Doc2Vec([document], size = 100, window = 1, min_count = 1, workers=1)
I have reduced the window size, and min_count so that they make sense for the given input. Also go through this nice tutorial on Doc2Vec, if you haven't already.