I want to take the weights of some layers - not all, as the architectures differ - from model_trained and initializes model_untrained with it. How can I do this with Keras?
If you have a function create_model() which returns a Keras model (example), you can initialize its weights like this:
from keras.models import load_model
model_untrained = create_model()
model_trained = load_model('trained_model.h5')
extracted_weights = model_trained.layers[0].get_weights()
model_untrained.layers[0].set_weights(extracted_weights)
Related
I have a pretrain model (vgg16) from keras. I'm trying to add BatchNormalization layer after every conv2d by looping. However, it seems that I could not all of them together. Here is my code.
from keras.applications import VGG16
from keras.layers import BatchNormalization, Input
from keras.models import Model
input_tensor = Input(shape=(256, 256, 3))
pretrain = VGG16(weights="imagenet", include_top=False, input_tensor=input_tensor)
model = pretrain.layers[0].input
for i in range(len(pretrain.layers)):
x = pretrain.layers[i].output
if "conv" in pretrain.layers[i].name:
x = pretrain.layers[i].output
x = BatchNormalization(axis=-1)(x)
model = Model(input=model.input, output=x)
May I have your suggestions? Thank you in advance
You should build a new model by copying layers from VGG and adding your own where you want.
The problem now is that the conv. layers are connected to the batch norm but which are not connected to anything.
When I read the Keras document I find a term called functional API.
What is the meaning of functional API in Keras?
Could anyone help you to understand the basic and significant terms in Keras?
Thanks
It is a way to create your models. One can use sequential models (tutorial here):
For example:
from keras.models import Sequential
from keras.layers import Dense, Activation
model = Sequential([
Dense(32, input_shape=(784,)),
Activation('relu'),
Dense(10),
Activation('softmax'),
])
And you can call it with input features. Second approach is functional (tutorial here). You call each layer with the next which allows you to have more flexibility when creating your model, for example:
from keras.layers import Input, Dense
from keras.models import Model
# This returns a tensor
inputs = Input(shape=(784,))
# a layer instance is callable on a tensor, and returns a tensor
output_1 = Dense(64, activation='relu')(inputs)
# you call layer with another layer to create model
output_2 = Dense(64, activation='relu')(output_1)
predictions = Dense(10, activation='softmax')(output_2)
# This creates a model that includes
# the Input layer and three Dense layers
model = Model(inputs=inputs, outputs=predictions)
model.compile(optimizer='rmsprop',
loss='categorical_crossentropy',
metrics=['accuracy'])
model.fit(data, labels) # starts training
You could also subclass Model, similar to what Chainer or PyTorch provide to the user, but it's used that heavily in Keras.
I am using Keras functional API to build a classifier and I am using the training flag in the dropout layer to enable dropout when predicting new instances (in order to get an estimate of the uncertainty). In order to get the expected response one needs to repeat this prediction several times, with keras randomly activating links in the dense layer, and of course it is computational expensive. Therefore, I would also like to have the option to not use dropout at the prediction phase, i.e., use all the network links. Does anyone know how I can do this? Following is a sample code of what I am doing. I tried to look if predict has any relevant parameter but does not seem like it does (?). I can technically train the same model without the training flag at the dropout layer, but I do not want to do this (or better I want to have a more clean solution, rather than having 2 different models).
from sklearn.datasets import make_circles
from keras.models import Sequential
from keras.utils import to_categorical
from keras.layers import Dense
from keras.layers import Dropout
import numpy as np
import keras
# generate a 2d classification sample dataset
X, y = make_circles(n_samples=100, noise=0.1, random_state=1)
n_train = 30
trainX, testX = X[:n_train, :], X[n_train:, :]
trainy, testy = y[:n_train], y[n_train:]
trainy = to_categorical(trainy)
testy = to_categorical(testy)
inputlayer = keras.layers.Input((2,))
d = keras.layers.Dense(500, activation = 'relu')(inputlayer)
d1 = keras.layers.Dropout(rate = .3)(d,training = True)
out = keras.layers.Dense(2, activation = 'softmax')(d1)
model = keras.Model(inputs = inputlayer, outputs = out)
model.compile(loss = 'categorical_crossentropy',metrics = ['accuracy'],optimizer='adam')
model.fit(x = trainX, y = trainy, validation_data=(testX, testy),epochs=1000, verbose=1)
# another prediction on a specific sample
print(model.predict(testX[0:1,:]))
# another prediction on the same sample
print(model.predict(testX[0:1,:]))
Running the above example I get the following output:
[[0.9230819 0.07691813]]
[[0.8222245 0.17777553]]
which is as expected, different class probabilities for the same input, since there is a random (de)activation of the links from the dropout layer.
Any suggestions on how I can enable/disable dropout at the prediction phase with the functional API?
Sure, you do not need to set the training flag when building the Dropout layer. After training your model you define this function:
mc_func = K.function([model.input, K.learning_phase()],
[model.output])
Then you call mc_func with your input and flag 1 to enable dropout, or 0 to disable it:
stochastic_pred = mc_func([some_input, 1])
deterministic_pred = mc_func([some_input, 0])
I studied Keras and created my first neural network model as the following:
from keras.layers import Dense
import keras
from keras import Sequential
from sklearn.metrics import accuracy_score
tr_X, tr_y = getTrainingData()
# NN Architecture
model = Sequential()
model.add(Dense(16, input_dim=tr_X.shape[1]))
model.add(keras.layers.advanced_activations.PReLU())
model.add(Dense(16))
model.add(keras.layers.advanced_activations.PReLU())
model.add(Dense(1, activation='sigmoid'))
# Compile the Model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
# Fit the Model
model.fit(tr_X, tr_y, epochs=1000, batch_size=200, validation_split=0.2)
# ----- Evaluate the Model (Using UNSEEN data) ------
ts_X, ts_y = getTestingData()
yhat_classes = model.predict_classes(ts_X, verbose=0)[:, 0]
accuracy = accuracy_score(ts_y, yhat_classes)
print(accuracy)
I am not sure about the last portion of my code, i.e., model evaluation using model.predict_classes() where new data are loaded via a custom method getTestingData(). See my goal is to test the final model using new UNSEEN data to evaluate its prediction. My question is about this part: Am I evaluating the model correctly?
Thank you,
Yes, that is correct. You can use predict or predict_classes to get the predictions on test data. If you need the loss & metrics directly, you can use the evaluate method by feeding ts_X and ts_y.
y_pred = model.predict(ts_X)
loss, accuracy = model.evaluate(ts_X, ts_y)
https://keras.io/models/model/#predict
https://keras.io/models/model/#evaluate
Difference between predict & predict_classes: What is the difference between "predict" and "predict_class" functions in keras?
I trained a model with Resnet3D and I want to extract the neurons of a layer. I plan to use them with the SVM classifier. How can I extract these weights and put them to the numpy array?
Load the weights by keras
model = Resnet3DBuilder.build_resnet_18((128, 96, 96, 3), nClass[0])
model.load_weights('drive/app/models/3d_resnet_modelq.hdf5')
extract a layer
dns = model.layers[-1].output
now what should i do?
If you just want to visualise the features, in pure Keras you can define a Model with the desired layer as output:
from keras.models import Model
model_cut = Model(inputs=model.inputs, output=model.layers[-1].output)
features = model_cut.predict(x) # Assuming you have your images in x
Note that in order for this to work, model must have been compiled at least once.