I have tried to write my own custom loss function in Keras. But writing complex loss functions are normally requiring deep knowledge of TensorFlow and Keras Backend. Do I need to study them to write my own loss function or is there an alternative method for writing custom loss functions for neural networks in Keras?
My loss function requires the probability of prediction of all classes and some way to point out the probability corresponding to the label class.
Related
In the fully connected hidden layer of Keras embedding, what is the activation function leveraged? I'm either misunderstanding the concept of this class or unable to find documentation. I understand that it is encoding from word to real-valued vector of dimension d via answers like the below on stackoverflow:
Embedding layers in Keras are trained just like any other layer in your network architecture: they are tuned to minimize the loss function by using the selected optimization method. The major difference with other layers, is that their output is not a mathematical function of the input. Instead the input to the layer is used to index a table with the embedding vectors [1]. However, the underlying automatic differentiation engine has no problem to optimize these vectors to minimize the loss function...
In my network, I have a word embedding portion that is then linked to a larger network that is predicting a binary outcome (e.g., click yes/no). I understand that this Keras embedding is not operating like word2vec because here my embedding is being trained and updated against my end cross-entropy function. But, there is no mention of how the embedding fully-connected layer is activated. Thanks!
I would like to know how to add in custom weights for the loss function in a binary or multiclass classifier in Keras. I am using binary_crossentropy or sparse_categorical_crossentropy as the baseline and I want to be able to choose what weight to give incorrect predictions for each class.
For multiple classes one should use not binary but categorical crossentropy.
Consider using custom loss function as described here: Custom loss function in Keras
In Keras I often see people compile a model with mean square error function and "acc" as metrics.
model.compile(optimizer=opt, loss='mse', metrics=['acc'])
I have been reading about acc and I can not find an algorithm for it?
What if I would change my loss function to binary crossentropy for an example and use 'acc' as metrics? Would this be the same metrics as in first case or Keras changes this acc based on loss function - so binary crossentropy in this case?
Check the source code from line 375. The metric_fn change dependent on loss function, so it is automatically handled by keras.
If you want to compare models using different loss function it could in some cases be necessary to specify what accuracy method you want to grade your model with, such that the models actually are tested with the same tests.
I am having a problem at hand which optimizes a loss function that is not a function of y_pred and y_true . After going through the Keras documentation , I found out that all the custom loss functions must be a function of both y_pred and y_true.
Is there any alternate way of implementing my kind of loss function in Keras?
No.This is the flaw of keras.
If you want to use that type of loss function, then the basic stochastic gradient descent scheme won't work.Many concepts such as batch size will disappear and that will be a substantive change so keras does not allow you to do so.
How to write a categorization accuracy loss function for keras (deep learning library)?
Categorization accuracy loss is the percentage of predictions that are wrong, i.e. #wrong/#data points.
Is it possible to write a custom loss function for that?
Thanks.
EDIT
Although Keras allows you to use custom loss function, I am not convinced anymore that using accuracy as loss makes sense. First, the network's last layer will typically be soft-max, so that you obtain a vector of class probabilities rather than the single most likely class. Second, I fear that there will be issues with gradient computation due to lack of smoothness of accuracy.
OLD POST
Keras offers you the possibility to use custom loss functions. To get the accuracy loss, you can take inspiration from the examples that are already implemented. For binary classification, I would suggest the following implementation
def mean_accuracy_error(y_true, y_pred):
return K.mean(K.abs(K.sign(y_true - y_pred)), axis=-1)