I want to change weights in a certain kernels in a my saved CNN model. How can I change the values in specific kernels and save to a new model?
You can torch.load the weights you saved. You should get a state_dict dictionary in which the weights are stored. Use the state_dict keys to locate the weights you wish to change, modify them and then torch.save the modified state_dict (better use different filename ;).
Related
I am using model.save("cnn.model") and model.save("cnn.h5") to save the model after training.
What is the difference of the saving the model in 2 different extensions?
File name, which includes the extension, doesn't matter. Whatever it is, Keras will save a HDF5 formatted model into that file.
Doc: How can I save a Keras model?
You can use model.save(filepath) to save a Keras model into a single
HDF5 file which will contain:
the architecture of the model, allowing to re-create the model
the weights of the model
the training configuration (loss, optimizer)
the state of the optimizer, allowing to resume training exactly where you left off.
How can I use the weights of a pre-trained network in my tensorflow project?
I know some theory information about this but no information about coding in tensorflow.
As been pointed out by #Matias Valdenegro in the comments, your first question does not make sense. For your second question however, there are multiple ways to do so. The term that you're searching for is Transfer Learning (TL). TL means transferring the "knowledge" (basically it's just the weights) from a pre-trained model into your model. Now there are several types of TL.
1) You transfer the entire weights from a pre-trained model into your model and use that as a starting point to train your network.
This is done in a situation where you now have extra data to train your model but you don't want to start over the training again. Therefore you just load the weights from your previous model and resume the training.
2) You transfer only some of the weights from a pre-trained model into your new model.
This is done in a situation where you have a model trained to classify between, say, 5 classes of objects. Now, you want to add/remove a class. You don't have to re-train the whole network from the start if the new class that you're adding has somewhat similar features with (an) existing class(es). Therefore, you build another model with the same exact architecture as your previous model except the fully-connected layers where now you have different output size. In this case, you'll want to load the weights of the convolutional layers from the previous model and freeze them while only re-train the fully-connected layers.
To perform these in Tensorflow,
1) The first type of TL can be performed by creating a model with the same exact architecture as the previous model and simply loading the model using tf.train.Saver().restore() module and continue the training.
2) The second type of TL can be performed by creating a model with the same exact architecture for the parts where you want to retain the weights and then specify the name of the weights in which you want to load from the previous pre-trained weights. You can use the parameter "trainable=False" to prevent Tensorflow from updating them.
I hope this helps.
I have trained my model and saved it. For plotting results against that model I loaded the model and now I want to extract history from that model.
saved model
model = load_model('model_pre_ep3_valloss0.360.h5')
How do I extract accuracy, loss etc plot from there?
That information is not saved a Keras HDF5 model file, so if you did not save it in another file, it is effectively lost.
One simple way to save the loss/accuracy history is to use the CSVLogger callback, more information in the keras docs.
Yesterday night I let a neural network model training and that took time, so I thought to add a statement to save weights model.save_weights('first_try.h5')
Now as I had the file, I want to benefit saved file.
Prediction is like
pred=model.predict_generator(test_generator, steps=4124, verbose=1)
If you saved your model's weights you can load using load_weights method. But first you have to define your model structure.
e.g.
model = method_to_create_the_model()
model.load_weights("path_to_weight_file")
I am working on a project in which I need to edit individual weights and biases.
Is there any way to actually get access to the layers weights and biases so I can edit them manually?
from tf.layers.dense()
So far I have created my own model and stored the biases outside like so:
for _ in range(population_size):
hidden_layer.append(tf.Variable(tf.truncated_normal([11, 20])))
output_layer.append(tf.Variable(tf.truncated_normal([20, 9])))
population.append([hidden_layer, output_layer])
I am then trying to feed population into the model using feed dict. It's turning out to be a real hell because I cannot feed them into the models because of the shapes of the Variables are not the same.
Is there any native support for getting the weights from the dense layer?
From the keras doc:
All Keras layers have a number of methods in common:
layer.get_weights(): returns the weights of the layer as a list of Numpy arrays.
layer.set_weights(weights): sets the weights of the layer from a list of Numpy arrays (with the same shapes as the output of get_weights).
You can easily access all layers inside your model with yourmodel.layers.