tryint to use the test after I load the model
net = net.load_state_dict(torch.load(PATH))
net.eval()
but this spit the error
net.eval() AttributeError: '_IncompatibleKeys' object has no attribute 'eval'
here you don't need to assign net.load_state_dict to net
net = net.load_state_dict(torch.load(PATH))
just used
net.load_state_dict(torch.load(PATH))
net.eval()
more see Recommended approach for saving a model
Related
I am dumping my pretrained doc2vec model using below command
model.train(labeled_data,total_examples=model.corpus_count, epochs=model.epochs)
print("Model Training Done")
#Saving the created model
model.save(project_name + '_doc2vec_vectorizer.npz')
vectorizer=CountVectorizer()
vectorizer.fit(df[0])
vec_file = project_name + '_doc2vec_vectorizer.npz'
**pickle.dump(vectorizer, open(vec_file, 'wb'))**
vdb = db['vectorizers']
and then I am loading Doc2vec model using below command in another function
loaded_vectorizer = pickle.load(open(vectorizer, 'rb'))
and then getting the error CountVectorizer has no attribute _load_specials on below line i.e model2
model2= gensim.models.doc2vec.Doc2Vec.load(vectorizer)
The gensim version being used by me is 3.8.3 as I am using the LabeledSentence class
The .load() method on Gensim model classes should only be used with objects of exactly that same class that were saved to file(s) *using the Gensim .save() method.
Your code shows you trying to use Doc2Vec.load() with the vectorizer object itself (not a file path to the previously-saved model), so the error is to be expected.
If you actually want to pickle-save & then pickle-load the vectorizer object, be sure to:
use a different file path than you did for the model, or you'll overwrite the model file!
use pickle methods (not Gensim methods) to re-load anything that was pickle-saved
I have a python script that trains and then tests a CNN model. The model weights/parameters are saved after testing through the use of:
checkpoint = {'state_dict': model.state_dict(),'optimizer' :optimizer.state_dict()}
torch.save(checkpoint, path + filename)
After saving I immediately load the model through the use of a function:
model_load = create_model(cnn_type="vgg", numberofclasses=len(cases))
And then, I load the model weights/parameters through:
model_load.load_state_dict(torch.load(filePath+filename), strict = False)
model_load.eval()
Finally, I feed this model the same testing data I used before the model was saved.
The problem is that the testing results are not the same when I compare the testing results of the model before saving and after loading. My hunch is that due to strict = False, some of the parameters are not being passed through to the model. However, when I make strict = True. I receive errors. Is there a work around this?
The error message is:
RuntimeError: Error(s) in loading state_dict for CNN:
Missing key(s) in state_dict: "linear.weight", "linear.bias", "linear 2.weight", "linea r2.bias", "linear 3.weight", "linear3.bias". Unexpected key(s) in state_dict: "state_dict", "optimizer".
You are loading a dictionary containing the state of your model as well as the optimizer's state. According to your error stack trace, the following should solve the issue:
>>> model_state = torch.load(filePath+filename)['state_dict']
>>> model_load.load_state_dict(model_state, strict=True)
i tried several ways but i get the same error so i tried the same as https://androidkt.com/saved-keras-model-to-predict-text-from-scratch/
txt="السلام عليكم"
seq= loaded_tokenizer.texts_to_sequences([txt])
padded = pad_sequences(seq, maxlen=max_len)
pred = model.predict_classes(padded)
print(pred)
this is how i loaded the tokenizer
with open('pic2.pickle', 'rb') as handle:
loaded_tokenizer = pickle.load(handle)
this is my first time working with text models and i am not sure how to solve this issue
This error occur in Maxpooling stage while i train my CNN model
Error: Attribute Error: 'None Type' object has no attribute 'current'. Please help.
model = model.add(MaxPooling2D(pool_size=(2,2),input_shape=(48,48,1)))
The question is missing some info, but I think I can see what's going on.
Assuming that model was at some point a tf.models.Sequential() I guess you did something like:
model = models.Sequential()
model = model.add(...)
model = model.add(MaxPooling2D(pool_size=(2,2),input_shape=(48,48,1)))
However, that's not quite how model.add(..) works. Instead of returning a new model, it modifies the existing model.
Instead you should do something like:
model = models.Sequential() # create a first model
model.add(...) # add things to the existing model
model.add(MaxPooling2D(pool_size=(2,2),input_shape=(48,48,1)))
I'm trying to export an FCN from torchvision using the following code:
model= models.segmentation.fcn_resnet101(pretrained=True, progress=True, num_classes=21, aux_loss=None)
model.eval()
x = torch.randn(1,3,512,512)
torch_out = model(x)
torch.onnx.export(model, x, "seg_rn.onnx",
export_params=True,
opset_version=11,
do_constant_folding=True,
verbose=True
)
When exporting the model, I need minimum opset 11 to support the way pytorch's interpolation works, and this is confirmed by the output of the onnx model when running in the python onnx runtime.
Running in the python onnx runtime is fine, but when I load the model in onnxjs, like this:
var session = new InferenceSession();
const modelURL = "./models/seg_rn.onnx";
await session.loadModel(modelURL);
I get Uncaught (in promise) TypeError: cannot resolve operator 'Shape' with opsets: ai.onnx v11
If I go and create my own copies of bits from torchvision.models.segmentation, I can get rid of the error about Shape (by speciying a static shape for the input and telling the interpolation what the resizing factor should be), but then I get basically the same error: Uncaught (in promise) TypeError: cannot resolve operator 'MaxPool' with opsets: ai.onnx v11 but this time in reference to MaxPool. Ignoring tests and outputting with opset v10 results in a loadable model, but one which will be incorrect.
What is going on? Is there any way forward, or am I basically stuck?