I am training a face recognition model using Fisher Face algorithm using OpenCV library and Python language.
fisherFace = cv2.face.FisherFaceRecognizer_create()
fisherFace.train(imagefaceList, np.array(labelsIndexList))
I want to save this model in file/memory. In other word i want to save 'fisherface' object. I have tried pickle module for saving this object using this. I am not able to pickle and unpickle this object.Code is as below:
class test(object):
def __init__(self, a):
self.a = a
def pickle_test(t):
print('pickling a test instance...')
return test, (t.a,)
copyreg.pickle(test, pickle_test)
t = test(f)
t1 = copy.copy(t)
t2 = pickle.dumps(t)
Is there any way available that saves trained model for fisher face algorithm and use it at other place by loading same model for face recognition?
FaceRecognizer class has a save method, that stores a xml/yml file into the disk that can be loaded with the load method.
Here is the class method list.
So, you should be able to do
fisherFace.save("model.xml")
To save model to file.
https://docs.opencv.org/3.0-last-rst/modules/face/doc/facerec_api.html
To save a face recognizer once it has been trained, you can use the write method and save
the state as a yaml or an xml file.
To use the saved model in the next iteration of the program, call the read method of the recognizer and pass the location of the file where the state was saved.
Related
Having followed Chris McCormick's tutorial for creating a BERT Fake News Detector (link here), at the end he saves the PyTorch model using the following code:
output_dir = './model_save/'
if not os.path.exists(output_dir):
os.makedirs(output_dir)
# Save a trained model, configuration and tokenizer using `save_pretrained()`.
# They can then be reloaded using `from_pretrained()`
model_to_save = model.module if hasattr(model, 'module') else model
model_to_save.save_pretrained(output_dir)
tokenizer.save_pretrained(output_dir)
As he says himself, it can be reloaded using from_pretrained(). Currently, what the code does is create an output directory with 6 files:
config.json
merges.txt
pytorch_model.bin
special_tokens_map.json
tokenizer_config.json
vocab.json
So how can I use the from_pretrained() method to load the model with all of its arguments and respective weights, and which files do I use from the six?
I understand that a model can be loaded as such (from PyTorch documentation):
model = TheModelClass(*args, **kwargs)
model.load_state_dict(torch.load(PATH))
model.eval()
but how can I make use of the files in the output directory to do this?
Any help is appreciated!
I save the checkpoint and manager of my model pretrained (in python file named train), I open an other file and I would like to restore or load the saved checkpoint and manager in order to finetune my model like below:
checkpoint = tf.train.Checkpoint(step=tf.Variable(1), optimizer=None)
manager = tf.train.CheckpointManager(checkpoint, pretrain_save_path, max_to_keep=3)
I find this error
ValueError: `Checkpoint` was expecting a trackable object (an object derived from `TrackableBase`), got None. If you believe this object should be trackable (i.e. it is part of the TensorFlow Python API and manages state), please open an issue.
The tf.train.Checkpoint() receives kwargs paras which are key-object. And the object must be a trackable object(None is not a trackable object). So you only need delete , optimizer=None
Using Python 3.6 with Pytorch 1.3.1. I have noticed that some saved nn.Modules cannot be loaded when the whole module is being imported into another module. To give an example, here is the template of a minimum working example.
#!/usr/bin/env python3
#encoding:utf-8
# file 'dnn_predict.py'
from torch import nn
class NN(nn.Module):##NN network
# Initialisation and other class methods
networks=[torch.load(f=os.path.join(resource_directory, 'nn-classify-cpu_{fold}.pkl'.format(fold=fold))) for fold in range(5)]
...
if __name__=='__main__':
# Some testing snippets
pass
The whole file works just fine when I run it in the shell directly. However, when I want to use the class and load the neural network in another file using this code, it fails.
#!/usr/bin/env python3
#encoding:utf-8
from dnn_predict import *
The error reads AttributeError: Can't get attribute 'NN' on <module '__main__'>
Does loading of saved variables or importing modules happen differently in Pytorch than other common Python libraries? Some help or pointer to the root cause will be really appreciated.
When you save a model with torch.save(model, PATH) the whole object gets serialised with pickle, which does not save the class itself, but a path to the file containing the class, hence when loading the model the exact same directory and file structure is required to find the correct class. When running a Python script, the module of that file is __main__, therefore if you want to load that module, your NN class must be defined in the script you're running.
That is very inflexible, so the recommended approach is to not save the entire model, but instead just save the state dictionary, which only saves the parameters of the model.
# Save the state dictionary of the model
torch.save(model.state_dict(), PATH)
Afterwards, the state dictionary can be loaded and applied to your model.
from dnn_predict import NN
# Create the model (will have randomly initialised parameters)
model = NN()
# Load the previously saved state dictionary
state_dict = torch.load(PATH)
# Apply the state dictionary to the model
model.load_state_dict(state_dict)
More details on the state dictionary and saving/loading the models: PyTorch - Saving and Loading Models
I have an onnx graph/model that has big constants in it, so it is taking a lot of time to load it and parse it. Can I "strip" the data from the graph, so I inspect the graph nodes without its data ?
Initializer is one of the field in GraphProto. You should be able to clear initializer field with simple python script. I haven't tested the following code but should be something like this:
import onnx
def clear_initializer(model_path):
model = onnx.load_model(model_path)
model.graph.ClearField('initializer')
onnx.save_model(model)
references:
https://developers.google.com/protocol-buffers/docs/reference/python/google.protobuf.message.Message-class
https://github.com/onnx/onnx/blob/2e7099ee7c37b196c197c9a084a97698a41da232/onnx/init.py
After I call this function like this:
from tensorflow.contrib import learn
#----------------------------------------
#Do some process here
#----------------------------------------
classifier = learn.Estimator(model_fn=bag_of_words_model,model_dir='F:/data')
classifier.fit(feature_train, target_train, steps=1000)
I will have some file in my folder "F:/data" like this
And I wonder Do I have anyway to reuse this model ? Like move to new computer and use this to predict new data. Sorry for my bad English. Thanks for all the answers!! Hope you all have a nice day.
In a script where you want to reuse your model, redefine/import bag_of_words_model again and define
classifier_loaded = learn.Estimator(model_fn=bag_of_words_model, model_dir='F:/data')
Tensorflow will reload the graph and import the weights into the graph and you can just use it as
classifier_loaded.predict(input_fn=input_fn)
or continue the training with
claissifier_loaded.fit(feature_train, target_train)