how to use pre-trained model as discriminator in gan? python code
I want to use dc-gan for data augmentation and pre-trained neural network like alexnet vgg-16 as classification.
Related
after making the graph embedding with Doc2vec, I want to make classification with keras, do I have to make embedding layer and put it as input to neural network or I directly use the embedding and split it into training and testing? also did the embedding layer improves the accuracy of neural network or not
i would like to know when people say pretrained bert model, is it only the final classification neural network is trained
Or
Is there any update inside transformer through back propagation along with classification neural network
During pre-training, there is a complete training if the model (updation of weights). Moreover, BERT is trained on Masked Language Model objective and not classification objective.
In pre-training, you usually train a model with huge amount of generic data. Thus, it has to be fine-tuned with the task-specific data and task-specific objective.
So, if your task is classification on a dataset X. You fine-tune BERT accordingly. And now, you will be adding a task-specific layer (classification layer, in BERT they have used dense layer over [CLS] token). While fine-tuning, you update the pre-trained model weights as well as the new task-specific layer.
I am building a CNN using keras and I know that I can save the weights using model.save_weights() but how can I save the biases of the neural network? Am I missing something or is it that Keras just doesn't provide biases?
How do I use a pre-trained BERT model like bert-base-uncased as weights in the Embedding layer in Keras?
Currently, I am generating word embddings using BERT model and it takes a lot of time. And I am assigning those weights like in the cide shown below
model.add(Embedding(307200, 1536, input_length=1536, weights=[embeddings]))
I searched on internet but the method is given in PyTorch. I need to do it in Keras. Please help.
Is there a way I can train an autoencoder model using a pre-trained model like ResNet?
I'm trying to train an autoencoder model with input as an image and output as a masked version of that image.
Is it possible to use weights from a pretrained model here?
Yes! you can definitely do transfer learning using a pre-trained network, i.e. ResNet50 as the encoder in an autoencoder. For reference, check out the following link. https://github.com/hsinyilin19/ResNetVAE
From what I know, there is no proven method to do this. I'd train the autoencoder from scratch.
In theory, if you find a pre-trained CNN which does not use max pooling, you can use those weights and architecture for the encoder stage in your autoencoder. You can also extract features from a pre-trained model and concatenate/merge them to your autoencoder. But the value add is not clear, and the architecture might become overly complex.