I'd like to use the pruned model and weight but,
whenever I load the model or weight, this error messages keep popping up... My weight and model's files:
Error messages
The reason why you have this error is that some weights present in the file you are trying to load have no corresponding parts in the initialized model (since it has been pruned). One quick way to go around that is to silent the warnings with the strict=False option:
model.load_state_dict(state_dict, strict=False)
You can actually store the result of that call and log the missing and unexpected modules found in the file.
Related
I got one question about torch.
I load pre-training model like:
model_name = "bert-base-uncased"
model = BertTokenizer.from_pretrained(model_name)
and I read To train the model, you should first set it back in training mode with model.train().
but I don't understand how it does work. when I read document of from_pretrained(), there isn't any explanation about train().
How it works?
.train() is a method of torch.nn.Module. It notifies the module to switch to the training mode, see documentation. What exactly happens under the hood is up to the actual Module, in many modules it doesn't change anything, so without knowing your network we cannot say what examctly happens. But for instance in torch.nn.BatchNormNd it has an effect.
I am playing around with code from this Github repository https://github.com/jindongwang/Pytorch-CapsuleNet.
After training the model for 5 epochs, I got an accuracy of 99.2% on the test dataset. So I saved model using the following code:
torch.save(capsule_net.state_dict(),"capsnet_mnist_state.pt")
I tried loading the model back in another machine with the below code:
capsnet = CapsNet(Config())
capsnet.load_state_dict(torch.load('capsnet_mnist_state.pt'))
capsnet.eval()
Now the model predicts the 0 has the output for every input. Is there anything wrong with way I saved the model or loaded the model?.
I don't think there is anything wrong with the way you saved or the way you load your model.
But I happened to see the same problem because I didn't initialise some parameters, so they were stuck to 'None' even after loading the state dict.
I would suggest to look into the loaded state_dict to see if any parameters makes no sense.
Hope this help.
I am using the class api to subclass a model based on keras.models.Model.
Is there some trick to getting the save_weights working?
Am seeing errors like
ValueError: Layer #0 (named "dense_S") expects 0 weight(s), but the saved weights have 2 element(s).
Am trying by_name=True and False.
EDIT: it seems that calling predict once, with ANY data, is needed to build the layers for some reason. It would be interesting to here a proper explanation from anyone who knows more.
I need to get several outputs from several layers from keras model instead of getting the output from the last layer.
I know how to adjust the code according to what I need, but don't know how I can use it within keras application. I meant how can I import it then. Do I need to infall the setup.py at keras-application again. I did so but nothing happen. My changes doesn't applied. Is there any different way to get the outputs from different layer within the model?
Actually the solution was so simple, you need just to call the output of specified layer.
new_model= tf.keras.Model(base_model.input, base_model.get_layer(layer_name).output)
I tried converting a pytorch model into deployment (ios) so I'm converting a model to either cafffe or Onnx (https://discuss.pytorch.org/t/typeerror-forward-missing-2-required-positional-arguments-cap-lens-and-hidden/20010/8). My initial error was only having 'one layer" inside an lstm yet I encountered another problem.
I tried implementing two layers (nlayers=2) by instantiating a new rnn object called text_encoder
enter image description here
Yet I'm given an error of some key(s) inside the state_dict is missing. This error doesn't occur for having one layer yet but I do get an error during the conversion (https://discuss.pytorch.org/t/typeerror-forward-missing-2-required-positional-arguments-cap-lens-and-hidden/20010/8)
I'm not sure if this is happening because the model I loaded only had one layer or another problem. How can I recover missing key's while adding a new layer? Or is that impossible? Original post (https://discuss.pytorch.org/t/missing-key-s-in-state-dict/20154) and (https://discuss.pytorch.org/t/typeerror-forward-missing-2-required-positional-arguments-cap-lens-and-hidden/20010/11)
enter image description here