Here is the code:
finetuned_model = hub.load(hub_url)
shared_embedding_layer = hub.KerasLayer(finetuned_model,trainable=True)
"Fine Tune Network"
model.fit([left_inputs,right_inputs],similiarity,epochs=1,callbacks=[stopAtLossValue()])
The network looks like this.
After each epoch completes, my finetuned_model is updated slightly. What I would like to do is ideally two things:
Use this finetuned_model after each epoch
Save this finetuned_model after each epoch
This is interesting as I'm not actually interested in saving the model that is training. Is there are a way for me to do this?
Disclaimer: I'm not all that experience in Deep Learning
Related
I know that you can log metrics as your experiment progresses. For example the training loss over epochs for your DL model.
I was wondering if it was possible to do something similar for text. In my particular case I have a text model that generates some example text after each epoch and I wish to see what it's like. For example:
Epoch 1:
tHi is RubisH
Epoch 2:
Ok look slight better
Epoch 3:
I can speak English better than William Shakespeare
The workaround I can think of is to log this to a text file and push that as an artifact in mlflow. Was wondering if there was something else more native to mlflow.
You can use log_param/log_params for that. For long texts maybe it's better to use log_text instead...
I am training a Variational Auto-Encoder (VAE) with unlabelled input images. My interest here is to visualize the 3 classes of unlabelled data in the latent space.
I set the latent dimension to 128 and further use PCA to visualize in 2D.
I am new to this and seeking some clarity on this.
Firstly, while I train the network, I see the accuracy and validation accuracy being displayed. Since the input images to the network are not labeled, I wonder what exactly is accuracy computed based on. (According to what I have read, accuracy = number of samples correctly predicted/ total number of samples).
Secondly, my training code looks like this:
vae.compile(optimizer='rmsprop', loss=kl_reconstruction_loss, metrics=['accuracy'])
history=vae.fit_generator(X_train, epochs=15,
validation_data=next(X_val), validation_steps=5,
callbacks=[ReduceLROnPlateau(monitor='val_loss', factor=0.5, verbose=2,
patience=4, cooldown=1, min_lr=0.0001)])
During training, validation loss goes to zero and accuracy to 1 in 2 epochs.
Here, for three classes of unlabelled data, the trained network does not very well cluster 3 different classes. Not very clear about why is accuracy shooting up to 1 and loss to 0 while the network is not able to generalize very well on the test dataset
Epoch 1/2
2164/2164 [==============================] - 872s 403ms/step - loss: 6668.9662 - accuracy: 0.7253 - val_loss: 3785921.0000 - val_accuracy: 0.9982
Epoch 2/2
2164/2164 [==============================] - 869s 401ms/step - loss: 0.0000e+00 - accuracy: 1.0000 - val_loss: 3857374.2500 - val_accuracy: 0.9983
Any insights/suggestions?
The purpose of a VAE is to compress the input into a well-behaved latent representation and then use this latent representation to accurately reconstruct the input. There are two terms in the loss for a VAE, each one corresponding to one of the tasks in the first sentence. The first term in the loss - the KL divergence term - forces the latent representation to be drawn from a multidimensional unit Gaussian distribution. The second term in the loss makes the VAE accurately reconstruct the input. Typically people use something like the L2 loss between the input and the output (there are more fancy things to use, but L2 loss usually does OK).
Since the model is not performing classification, accuracy is not a good metric to use. A better way to see how accurately the VAE is reconstructing the input is to monitor something like mean squared error.
The output of your code (where training loss goes to 0 after 1 epoch) indicates that the model is overfitting the training data. Try regularizing your model (or using a less capacious one) or reducing the number of steps per training epoch so you can monitor the performance of your model on the validation data more often.
Also, your usage of next(X_val) is just grabbing one element from your validation set. You probably want to pass in the validation data generator, not a single element from the validation data. Removing the next() call will achieve that.
I am trying sentiment analysis of images.
I have 4 classes - Hilarious , funny very funny not funny.
I tried pre trained models like VGG16/19 densenet201 but my model is overfitting getting training accuracy more than 95% and testing around 30
Can someone give suggestions what else I can try?
Training images - 6K
You can try the following to reduce overfitting:
Implement Early Stopping: compute the validation loss at each epoch and a patience threshold for stopping
Implement Cross Validation: refer to Section Cross-validation in
https://cs231n.github.io/classification/#val
Use Batch Normalisation: normalises the activations of layers to
unit variance and zero mean, improves model generalisation
Use Dropout (either or with batch norm): randomly zeros some activations to incentivise use of all neurons
Also, if your dataset isn't too challenging, make sure you don't use overly complex models and overkill the task.
I am training blood cell images using chainer. While training the epoch details doesn't get updated and does not run the given set of epochs.
I want to understand the cause of this problem..
When the training is interrupted and restarted only a single epoch is updated and displayed..
I am not sure of the reason behind the problem..so I can't point towards a particular section of the code..whether it is the data pre-processing, or data feeding or the classifier/evaluator section.
You can see the whole code here...https://github.com/atom2k17/BloodCell-Chainer/blob/master/WithoutKerasDD-checkpoint.ipynb
After training epoch, main/loss, validation/loss, etc should be populated with values from each epoch..and each epoch should get updated after each epoch is finished.
Can you try modifying
valid_iter = iterators.SerialIterator(valid, batch_size)
to
valid_iter = iterators.SerialIterator(valid, batch_size, repeat=False, shuffle=False)?
Without repeat=False option, the iterator will not finish so
E.Evaluator(valid_iter, model_loss, device=gpu_id) never finish.
I ran my sample code using Keras.
model = Sequential([
BatchNormalization(axis=1, input_shape=(3,224,224))
Flatten(),
Dense(10, activation='softmax')])
model.compile(Adam(lr=1e-4), loss="categorical_crossentropy", metrics=['accuracy'])
model.fit_generator(batches, batches.nb_sample, nb_epoch=2,
validation_data=test_batches, nb_val_samples=test_batches.nb_sample)
It gave this output:
None
Epoch 1/2
500/500 [==============================] - 147s - loss: 2.2464 - acc: 0.3520 - val_loss: 6.4765 - val_acc: 0.1100
Epoch 2/2
500/500 [==============================] - 140s - loss: 0.8074 - acc: 0.7880 - val_loss: 3.8807 - val_acc: 0.1450
I'm not able to find the meaning of loss, acc, val_loss, val_acc. Any explanation or link to the doc will be helpful.
This is closest to what I'm looking for. In above code, I'm fitting the model. But it is also giving a validation accuracy. From which data set is this validation accuracy is calculated?
Loss is the objective function that you are minimizing to train a neural network. The loss value is the mean value of the loss function across batches in the training set. Accuracy (acc) is the mean accuracy across batches, also on the training set. Accuracy is just the fraction of samples in the dataset that the model classified correctly.
But the val metrics are computed on the full validation set, which is the dataset you passed on parameter validation_data. This is done to check for overfitting during training.
Regarding your first question: I respectfully recommend you familiarize yourself with the basic mechanics of a neural network or look into one of the many MOOCs i.e. this excellent one from fast.ai. This is also beyond the scope of this forum since it´s doesn´t seem to about programming.
Your validation accuracy is calculated from the data that you provide by setting the validation_data parameter in your model.fit_generator() function. In your case you have set it to test_batches which is methodically very likely not correct. You need to split your data into three sets: One for training, one for validation (and by that seeing the progress of your training in regard to unseen data and getting useful information for tuning your hyperparameters) and one data set for testing (to evaluate the final score of your model).
One more thing: nb_val_samples is not a parameter of fit_generator anymore. See documentation here.