I am implementing the Keras Variational Autoencoder (https://keras.io/examples/generative/vae/). During the training process, the total loss printed is not the sum of the reconstruction loss and the kl loss terms as it should be. Any suggestions on how to solve this problem?
I suspect that the issue is related with the loss trackers but I have no idea how to solve.
If you use the code mentioned in your question the loss shown in during training is the total loss because you return the total loss in the training step. However if you have changed something you should attach your code (in git please :) and show the losses during training.
currently i am working with LSTM and GRU Modells.
I applied them on a Multivariate Time Series Problem.
reset_random_seeds()
# design network weekly
model = Sequential()
#inputshape is using time stamps , feautures
model.add(LSTM(64,activation="relu" ,dropout=0.2, input_shape=(1, 19)))
model.add(Dense(20))
model.add(Dense(1))
model.compile(loss=root_mean_squared_error, optimizer='adam')
# fit network
history = model.fit(train_X, train_y, epochs=300, batch_size=258, validation_data=(test_X, test_y), verbose=2, shuffle=False)
# plot history
pyplot.plot(history.history['loss'], label='train')
pyplot.plot(history.history['val_loss'], label='test')
pyplot.legend()
pyplot.show()
enter image description here
The Loss Validation looks like this and the results are pretty good to but i wonder what this graphs means to me.
Results:
enter image description here
enter image description here
So at first this loss graph tells you how high the cumulated error for each example in your validation or training dataset is. So the loss value implies how good or bad your model is after each training iteration.
You loss graph shows us, that you have a much higher training loss than validation loss. This should be a good indicator, that your model at least might not be overfitting. On the other hand it also can be an indication, that your validation dataset might be very small or not that representativ. For example your validation dataset could be very skewed.
So we would have to look into your datasets a little deeper to fully interpret your loss graphes. If you can provide them I would be happy to have a look.
I am new at deep learning.
I have a dataset with 1001 values of human pose upper body. The model I trained for that has 4 Conv layers and 2 fully connected layers with ReLu and Dropout. This is the result I got after 200 iterations. Does anyone have any ideas about why the curve of training loss decreases in a sharp way?
I think probably I need more data, as my dataset concludes numerical values what do you think is the best data augmentation method I have to use here?
Going straight to the problem...
I am using Keras flow_from_directory to load the data for sound classification. Data_generator without any augmentation and shuffle =True and although most of my models have a very good accuracy (92%) and a small val_loss the confusion matrix shows that the model is not predicting the labels correctly
I have tried simple models and complex models with keras flow_from_directory and data_generator on UrbanSound8k dataset. Also tried batch normalization, bias and kernel regularizers to avoid overfitting.
The results look almost random.
I'm training the VGG16 CNN model with the "Imagenet" weights. There is no convergence when training the model from scratch. I have used a saved model after one epoch from another machine and started training on top of that. What could be the issue?
Using Learning Rate - 0.0001 / 0.001 ( Tried using both )
Training Images - RGB (1,00,000 images)
When training the model with Imagenet Pretrained weights from scratch there is no convergence. The training accuracy stands still (70%) after several epochs. The loss (88%) is not reducing.
When training the model with Imagenet Pretrained weights with a .h5 model ( Got from another machine after first epoch with the same configuration ) there was convergence. The accuracy raised epoch by epoch up to 90% & above and the loss reduced gradually.
What could be the actual reason?