I am building a model for my multiclass text classification problem which looks like this:
model = tf.keras.Sequential()
model.add(hub_layer)
for units in [128, 128, 64 , 32]: # automatically adding layers through a for loop
model.add(tf.keras.layers.Dense(units, activation='relu'))
model.add(tf.keras.layers.Dropout(0.3))
model.add(tf.keras.layers.Dense(18, activation='softmax'))
model.summary()
# Optimiser Function
model.compile(optimizer='adam',
loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
history = model.fit(train_data_f,
epochs=20,
validation_data=test_data_f,
verbose=1,
class_weight=class_weights
)
Once the build is done, I would like to do the following:
Save the model
Load the model and apply it to an unseen text (For Example: the column on which I would want to apply this model is df['Unseen_Text]
how can I achieve this?
Related
I am new to Keras and trying to write my first code. I want to understand what 'model.predict' should return. Consider a simple model below.
model = keras.Sequential()
model.add(keras.layers.Dense(12, input_dim=232, activation='relu'))
model.add(keras.layers.Dense(232, activation='relu'))
model.add(keras.layers.Dense(1, activation='sigmoid'))
# compile the keras model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit the keras model on the dataset
model.fit(vSignal, vLabels, epochs=15, batch_size=100 )
# evaluate the keras model
_, accuracy = model.evaluate(vSignal, vLabels)
print('Accuracy: %.2f' % (accuracy*100))
pred=model.predict(vSignalT)
Consider we train the "model" with "vSignal" and "vLabels" as shown above. Now consider that the accuracy of the model as given by model.evaluate is 100%. Now if we give same data 'vSignal' to 'model.predict' should we get the 'vLabels' return?
pred=model.predict(vSignalT) returns a numpy arrays of predictions.
each row consists of one of the vlabels that the model predicted.
for more information refer to here
save return value of fit function:
hist = model.fit(vSignal, vLabels, epochs=15, batch_size=100 );
then check the
hist.history["accuracy"]
I have trained a keras LSTM model. But after training, all i get is the final parameters of the models after training with 10 epochs and batch size=120. How can i get intermediate parameter after a batch keras.
Example: after 120 sample in each batch i can get the intermediate parameter of this step.
I have tried callback method and backend in keras, but i do not know how to get the
'''python
model = Sequential()
model.add(Embedding(max_features, 32))
#model.add(LSTM(32, return_sequences=True, input_shape=(1,texts.shape[0])))
model.add(LSTM(32))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['acc'])
history_ltsm = model.fit(texts_train, y_train, epochs=10, batch_size=120, validation_split=0.2)
'''
I expected the model run step by step based on each batch to show the intermediate parameters, but not the all epochs.
Thank you very much!
I'm using mobilenet v2 to train a model on my images. I've frozen all but a few layers and then added additional layers for training. I'd like to be able to train from an intermediate layer rather than from the beginning. My questions:
Is it possible to provide the output of the last frozen layer as the
input for training (it would be a tensor of (?, 7,7,1280))?
How does one specify training to start from that first trainable
(non-frozen) layer? In this case, mbnetv2_conv.layer[153].
What is y_train in this case? I don't quite understand how y_train
is being used during the training process- in general, when does the
CNN refer back to y_train?
Load mobilenet v2
image_size = 224
mbnetv2_conv = MobileNetV2(weights='imagenet', include_top=False, input_shape=(image_size, image_size, 3))
# Freeze all layers except the last 3 layers
for layer in mbnetv2_conv.layers[:-3]:
layer.trainable = False
# Create the model
model = models.Sequential()
model.add(mbnetv2_conv)
model.add(layers.Flatten())
model.add(layers.Dense(16, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(3, activation='softmax'))
model.summary()
# Build an array (?,224,224,3) from images
x_train = np.array(all_images)
# Get layer output
from keras import backend as K
get_last_frozen_layer_output = K.function([mbnetv2_conv.layers[0].input],
[mbnetv2_conv.layers[152].output])
last_frozen_layer_output = get_last_frozen_layer_output([x_train])[0]
# Compile the model
from keras.optimizers import SGD
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['acc'])
# how to train from a specific layer and what should y_train be?
model.fit(last_frozen_layer_output, y_train, batch_size=2, epochs=10)
Yes, you can. Two different ways.
First, the hard way makes you build two new models, one with all your frozen layers, one with all your trainable layers. Add a Flatten() layer to the frozen-layers-only model. And you will copy the weights from mobilenet v2 layer by layer to populate the weights of the frozen-layers-only model. Then you will run your input images through the frozen-layers-only model, saving the output to disk in CSV or pickle form. This is now the input for your trainable-layers model, which you train with the model.fit() command as you did above. Save the weights when you're done training. Then you will have to build the original model with both sets of layers, and load the weights into each layer, and save the whole thing. You're done!
However, the easier way is to save the weights of your model separately from the architecture with:
model.save_weights(filename)
then modify the layer.trainable property of the layers in MobileNetV2 before you add it into a new empty model:
mbnetv2_conv = MobileNetV2(weights='imagenet', include_top=False, input_shape=(image_size, image_size, 3))
for layer in mbnetv2_conv.layers[:153]:
layer.trainable = False
model = models.Sequential()
model.add(mbnetv2_conv)
then reload the weights with
newmodel.load_weights(filename)
This lets you adjust which layers in your mbnetv2_conv model you will be training on the fly, and then just call model.fit() to continue training.
I trained a neural network with 37 Inputs. It has around 85% accuracy. Is it possible for me to find out which Input has the most effect. I tried this code but I cannot figure out how to find most important Input
weights = model.layers[0].get_weights()[0]
biases = model.layers[0].get_weights()[1]
One possible solution is to wrap your model with keras.wrappers.scikit_learn and then use Recursive Feature elimination in scikit-learn:
def create_model():
# create model
model = Sequential()
model.add(Dense(512, activation='relu'))
model.add(Dense(512, activation='relu'))
model.add(Dense(10, activation='softmax'))
# Compile model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
model = KerasClassifier(build_fn=create_model, epochs=100, batch_size=128, verbose=0)
rfe = RFE(estimator=model, n_features_to_select=1, step=1)
rfe.fit(X, y)
ranking = rfe.ranking_.reshape(digits.images[0].shape)
# Plot pixel ranking
plt.matshow(ranking, cmap=plt.cm.Blues)
plt.colorbar()
plt.title("Ranking of pixels with RFE")
plt.show()
If you need to visualize weights see here.
I am following examples on this page: https://machinelearningmastery.com/use-word-embedding-layers-deep-learning-keras/
which trains a word embedding on the data using an Embedding layer, like below:
model = Sequential()
model.add(Embedding(100, 8, input_length=max_length))
model.add(Flatten())
model.add(Dense(1, activation='sigmoid'))
# compile the model
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['acc'])
# summarize the model
print(model.summary())
the model starts with learning a word embedding from data, for each word, creates a 8-dimension vector.
What I would like to do, is that after this embedding is learned, I want to alter the matrix (or vectors of each word), by adding two more dimensions appended to the end of each vector. I will have another process that computes the values for this two dimensions.
Is there anyway I can do this?
Many thanks in advance
Yes - it's possible. Try to do this using following procedure:
Extract weight matrix:
weight_matrix = model.layers[0].get_weights()[0] # Matrix shape (100, 8).
Append your vectors:
new_weight_matrix = your_append(weight_matrix)
# Be sure that new_weight_matrix has shape of (100, 10)
Build an adjusted copy of your model:
new_model = Sequential()
new_model.add(Embedding(100, 10, input_length=max_length)) # Notice a change
new_model.add(Flatten())
new_model.add(Dense(1, activation='sigmoid'))
(Optional) freeze layers: In case you want to freeze embedding set:
new_model = Sequential()
new_model.add(Embedding(100, 10, input_length=max_length
trainable=False)) # Notice a change
new_model.add(Flatten())
new_model.add(Dense(1, activation='sigmoid'))
Compile a new model:
new_model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['acc'])
After compile and fit, you need to replace the weights with the new ones:
new_model.layers[0].set_weights(new_weight_matrix)