Keras' just provide a very brief definition of it's functions like:
Available metrics
binary_accuracy
binary_accuracy(y_true, y_pred)
...
But I want a math formula style definition, does anyone knows where can I find the math definitions of all the functions?
Keras is open source project and you can find everything on github, for your questions, the metrics calculation code can be found here:
binary_accuracy
= https://github.com/fchollet/keras/blob/master/keras/metrics.py#L20
Related
I am trying to use a customized loss function for my NN. I've implemented all operations in torch and I have complex numbers among my data.
I get the error while training a NN:
RuntimeError: _th_addr_out not supported on CPUType for ComplexFloat
Do you know any possible solution to deal with it?
Well it seems Complex Autograd in PyTorch is currently in a prototype state, and the backward functionality for some of function is not included.
For example: torch.sign, which is used in the backward computation of torch.abs, is not defined for complex tensors. same for torch.mv. So I debugged my code line by line to find the functions which are not included, and replaced them with a customized function :)
Hope for a lot more functions to be included in the next release of PyTorch.
I've been looking around for a while now. I would like to know if it's possible to modify/customize the loss function of the spaCy textcategorizer.
I mean, when you want to distill a model (for instance BERT) and want to add a regression component in the loss function to optimize (regarding the probabilities of each class instead only the labels), I don't understand where I should look for. I tried to explore some spaCy code but there is only a function to get the loss.
If someone know where to look for to visualize the loss function and change it (by writing a subclass for instance) it would be nice !
Thanks
Arnault
SpaCy is ultimately built on top of thinc and therefore, if you want to do custom work, you should tinker with Thinc, not SpaCy. SpaCy typically allows you to initialize a pipe with a raw Thinc model.
Especially since SpaCy's philosophy is to provide one implementation that works well not necessarily a super customizable framework.
I am new to XGBoost and I am currently working on a project where we have built an XGBoost classifier. Now we want to run some feature selection techniques. Is backward elimination method a good idea for this? I have used it in regression but I am not sure if/how to use it in a classification problem. Any leads will be greatly appreciated.
Note: I have already tried permutation line importance and it has yielded good results! Looking for another method to evaluate the features in the model.
Consider asking your question on Cross Validated since feature selection is more about theory/practice than code.
What is your concern ? Remove "noisy" features who drive down your results, obtain a sparse model ? Backward selection is one way to do of course. That being said, not sure if you are aware of this but XGBoost computes its own "variable importance" values.
# plot feature importance using built-in function
from xgboost import XGBClassifier
from xgboost import plot_importance
from matplotlib import pyplot
model = XGBClassifier()
model.fit(X, y)
# plot feature importance
plot_importance(model)
pyplot.show()
Something like this. This importance is based on how many times a feature is used to make a split. You can then define for instance a threshold below which you do not keep the variables. However do not forget that :
This variable importance has been obtained on the training data only
The removal of a variable with high importance may not affect your prediction error, e.g. if it is correlated with another highly important variable. Other tricks such as this one may exist.
The "embedding" class documentation https://pytorch.org/docs/stable/nn.html says
max_norm (float, optional) – If given, will renormalize the embedding vectors to have a norm lesser than this before extracting.
1) In my model, I use this embedding class as a parameter, not just as an input (the model learns the embedding.) In this case, I assume every time when updates happen, the embedding gets renormalized, not only when it's initialized. Is my understanding correct?
2) I wanted to confirm 1) by looking at the source, but I couldn't find the implementation in pytorch embedding class. https://pytorch.org/docs/stable/_modules/torch/nn/modules/sparse.html
Can someone point me to the max_norm implementation?
If you see forward function in Embedding class here, there is a reference to torch.nn.functional.embedding which uses embedding_renorm_ which is in the cpp documentation here which means it is a cpp implementation. Some github search on pytorch repo pointed to this files (1, 2).
Answer to 1 is yes. Answer to 2 is above.
I'm using lightgbm.train with valid_sets, early_stopping_rounds and feval function for multiclass problem with "objective": "multiclass". I want to find best_iteration and best_score for my custom evaluation function. But it finds them for multi_logloss metrics, which is corresponding to specified objective. So the question is can I find in LightGBM best_iteration and best_score for my feval function and how?
This happens due to the fact that the objective function is included in the list of evaluation metrics by default. Early stopping in LightGBM happens based on any included metric. See a short summary and a link to another issue with a longer discussion in this LightGBM issue.
You can use objective:"multi_error", or also you can combine objectives as
objective: "multi_error", "multi_logloss"
Multi_error will directly focus on the accuracy.