I have a problem making a prediction using a pre-trained model that contains an encoder and decoder for handwritten text recognition.
What I did is the following:
checkpoint = torch.load("Model/SPAN/SPAN-PT-RA_rimes.pt",map_location=torch.device('cpu'))
encoder_state_dict = checkpoint['encoder_state_dict']
decoder_state_dict = checkpoint['decoder_state_dict']
img = torch.LongTensor(img).unsqueeze(1).to(torch.device('cpu'))
global_pred = decoder_state_dict(encoder_state_dict(img))
This generates this error:
TypeError: 'collections.OrderedDict' object is not callable
I would highly appreciate your help! ^_^
encoder_state_dict and decoder_state_dict are not the torch Models, but a collection (dictionary) of tensors that include pre-trained parameters of the checkpoint you loaded.
Feeding inputs (such as the input image you got transformed) to such collection of tensors does not make sense. In fact, you should use these stat_dicts (i.e., a collection of pre-trained tensors) to load them into the parameters of your model object that is mapped to the network. See torch.nn.Module class.
Related
I am working through the Barlow Twins tutorial with PyTorch Lightning and I am having trouble loading the encoder portion of the model using the checkpoint after training.
During model training, checkpoints are saved with ModelCheckpoint. In the tutorial, the author offers two options for then getting the encoder portion of the model with trained weights: 1) calling model.encoder (model has to have been trained in the active kernel for this to work) or 2) loading the trained model with:
ckpt_model = torch.load('[checkpoint name].ckpt')
And then calling
encoder = ckpt_model.encoder
I would like to be able to load the model/encoder from a saved checkpoint but when I try to do this, I get the error: AttributeError: 'dict' object has no attribute 'encoder'
This seems to make sense to me because the model is being loaded with a simple Torch loader rather than the lightning loader.
When I print the contents of ckpt_model using ckpt_model.keys() I get:
dict_keys(['epoch', 'global_step', 'pytorch-lightning_version', 'state_dict', 'loops', 'callbacks', 'optimizer_states', 'lr_schedulers']). The state_dict contains weights and biases for the encoder layer.
Then when I try loading the model with the PyTorch Lightning loader:
BarlowTwins.load_from_checkpoint('[checkpoint name].ckpt')
I get the error: TypeError: init() missing 4 required positional arguments: 'encoder', 'encoder_out_dim', 'num_training_samples', and 'batch_size'. I might be able to save those in the checkpoint with save_hyperparameters.
My question: how can I load the encoder portion of the model in the simple way possible? I only need the encoder portion of the model loaded.
Thanks so much for help in advance!
I am new in Pytorch. My question is: How do I apply transfer learning to a custom dataset? I am doing image segmentation on brain tumors. I can find examples which use U-net structure but I could not find examples using weights of the pre-trained models for a U-net image segmentation?
You could obtain pre-trained models in two ways:
Model weights or complete models shared in formats such .pt or .pth:
In this case, Saving and Loading Models is a good starting point. Copying from the tutorial there, you could load a model as
model = TheModelClass(*args, **kwargs)
model.load_state_dict(torch.load(PATH))
The other way is to load the model from torchvision. A list is available models is available at Torchvision Models. U-Net is not available yet. However, it is possible to load a pre-trained model as the encoder and write a separate decoder to form a U-Net with a pre-trained encoder.
In this case, the model object returned from the function calls shown in the API are already loaded with pretrained weights when pretrained=True.
For writing a custom dataloader, PyTorch data loaders may be a useful guide.
I built a LSTM model for text classification using Keras. Now I have new data to be trained. instead of appending to the original data and retrain the model, I thought of training the data using the model weights. i.e. making the weights to get trained with the new data.
However, irrespective of the volume i train, the model is not predicting the correct classification (even if i give the same sentence for prediction). What could be the reason?
Kindly help me.
Are you using the following to save the trained model?
model.save('model.h5')
model.save_weights('model_weights.h5')
And the following to load it?
from keras.models import load_model
model = load_model('model.h5') # Load the architecture
model = model.load_weights('model_weights.h5') # Set the weights
# train on new data
model.compile...
model.fit...
The model loaded is the exact same as the model being saved here. If you are doing this, then there must be something different in the data (in comparison with what it is trained on).
I just completed the implementation of A Guide to TF Layers: Building a Convolutional Neural Network for the MNIST data set. The training model successfully ran and gave accuracy of 97.3%.
However, the tutorial does not mention how to use this new trained model to supply own images and see the predictions. Does anyone know how to use the output of the training model to make predictions? I see in the tmp/mnist_convnet_model$ folder, there are some output files like .pbtxt , meta files and index files. But I can't find instructions to use them for making predictions on my own images.
y_pred = tf.nn.softmax(your_final_layer)
y_pred_cls = tf.argmax(y_pred, dimension=1)
and for prediction
feed_dict = {x: [your_image]}
classification = tf.run(y_pred_cls, feed_dict)
print classification
This applies to just about any model you create
I'm trying to modify Keras Siamese Network example to get image feature.
The problem is, how can I get image features? The output of last layer is only a number. What should I do to get the feature before euclidean_distance?
You can try to first train the model on the entire dataset and save it.
Load the model back again, now set the output layers to be processed_a and processed_b
Now call the model.predict() function on the entire dataset once again and you'll have the features for each image in the dataset.
Have a look at this
Hope this helps!
To get the embeddings from the Keras siamese network MNIST example after training:
model_a = Model(inputs=model.input, outputs=processed_a)
model_a.predict([tr_pairs[:, 0], tr_pairs[:, 1]])
I did it as follows (reference from my github post):
My trained siamese model looked like this:
siamese_model.summary()
Note that my newly redefined model is basically the same as the one highlighted in yellow
I then redefined my model which I wanted to use for extracting embeddings (It should be the same model you defined except now it will not have those multiple inputs like siamese) which looked like this:
siamese_embeddings_model = build_siamese_model(input_shape)
siamese_embeddings_model .summary()
Then I just extracted the weights from my trained siamese model and set them into my new model
embeddings_weights = siamese_model.layers[-3].get_weights()
siamese_embeddings_model.set_weights(embeddings_weights )
Then you can supply the new Image to extract the embeddings from the new model
vector = siamese.predict(image)
len(vector[0]) it will print 150 because of my fine dense layer (which are the output vector)