I save and load pytorch state_dict file and then evaluate and retrain the model. That works.
torch.save(net.state_dict(), path)
net.load_state_dict(torch.load(path))
However, when I modify the state_dict file (manually changing the values) after loading it and evaluating it, I receive the error:
RuntimeError: a leaf Variable that requires grad has been used in an in-place operation.
How to safely modify the state_dict and then retrain it without the error?
Related
I have created a Pytorch object from the class Sequential (see official page).
As they suggest, I am saving it using the command torch.save(model.state_dict(), PATH).
It seems that everything has worked fine, since when I use torch.load(PATH) in another file I get
an ordered Dict like
'0.weight': tensor([[ 0.1202, ...]]) ,
'0.bias': tensor([ 0.1422, ...]) ,
...
with all the shapes of the tensors being correct. However, when doing
model = Sequential()
model.load_state_dict(torch.load(PATH))
I get the error
RuntimeError: Error(s) in loading state_dict for Sequential:
Unexpected key(s) in state_dict: "0.weight", "0.bias", "2.weight", "2.bias", "4.weight", "4.bias".
When trying to load, the model you are trying to load into (model) is an empty Sequential object with no layers. On the other hand, looking at the error message, the state dictionary of the model you are trying to load from indicates that it has at least five layers, with the first, third, and fifth layers containing a weight and bias parameter. This is a mismatch since the corresponding layers and parameters do not exist in model.
To fix this, model should have the same architecture as the saved model you are trying to load. Since you say you saved the original model yourself, use an identical initialization step when creating model.
I am struggling with restoring a keras model from a .pb file. I have seen a couple of posts explaining how to do inference using a model saved in .pb format, but what I need is to load the weights into a keras.Model class.
I have a function that returns an instance of the said original model, but untrained. I want to restore the weights of that model from the .pb file. My goal is to then truncate the model and use the truncated model for other purposes.
So far the furthest I've gotten is using the tf.saved_model.load(session, ['serving'], export_dir) function to get a tensorflow.core.protobuf.meta_graph_pb2.MetaGraphDef object. From here I can access the graph_def attribute, and from there the nodes.
How can I go from that to getting the weights and then loading those into the instance of the untrained keras Model?
Maybe if that's not doable there is a way to "truncate" the graph_def somehow and then make inference using that?
I am using model.save("cnn.model") and model.save("cnn.h5") to save the model after training.
What is the difference of the saving the model in 2 different extensions?
File name, which includes the extension, doesn't matter. Whatever it is, Keras will save a HDF5 formatted model into that file.
Doc: How can I save a Keras model?
You can use model.save(filepath) to save a Keras model into a single
HDF5 file which will contain:
the architecture of the model, allowing to re-create the model
the weights of the model
the training configuration (loss, optimizer)
the state of the optimizer, allowing to resume training exactly where you left off.
I trained and saved a keras model in json format and saved the weights in h5 format. The only issue is that each time I run predict, I get different results. There is no randomness when loading the data, I simply load the data (without shuffling), load the keras model and weights, and then run predict. I don't run the compile command, since it's not needed. Something I was wondering is if the dropout layers I added to the model contribute to this, but I read that they are automatically disabled for prediction/evaluation stages. Any ideas on why this might be happening?
I want to change weights in a certain kernels in a my saved CNN model. How can I change the values in specific kernels and save to a new model?
You can torch.load the weights you saved. You should get a state_dict dictionary in which the weights are stored. Use the state_dict keys to locate the weights you wish to change, modify them and then torch.save the modified state_dict (better use different filename ;).