Good day, I am a student that is interested in NLP. I have come across the demo on AllenNLP's homepage, which stated that:
The model is a simple LSTM using GloVe embeddings that is trained on the binary classification setting of the Stanford Sentiment Treebank. It achieves about 87% accuracy on the test set.
Is there any reference to the sample code or any tutorial that I can follow to replicate this result, so that I can learn more about this subject? I am trying to obtain a Regression Output (Instead of classification).
I hope that someone can point me in the right direction.. Any help is much appreciated. Thank you!
AllenAI provides all code for examples and lib opensource on Git, including AllenNLP.
I found exactly how the example was run here: https://github.com/allenai/allennlp/blob/master/allennlp/tests/data/dataset_readers/stanford_sentiment_tree_bank_test.py
However, to make it a Regression task, you'll have to tweak directly on Pytorch, which is the underlying technology for AllenNLP.
Related
so this is a specific question involving two Tensorflow text classification tutorials on tensorflow.org. Sorry if this is the wrong place to ask.
Basically, there are two tutorials, one is "Classify Text with BERT" https://www.tensorflow.org/text/tutorials/classify_text_with_bert
And the other is "Fine-tuning a BERT model"
https://www.tensorflow.org/text/tutorials/fine_tune_bert
In these two tutorials, it describes preprocessing data. In "Classify Text with BERT", they use a preprocessing model provided by Tensorflow Hub, but in "Fine-tuning a BERT model", they implement python code which tokenizes the data and encodes it and some other stuff. Basically, it seems like the latter method is a lot more complicated than the former.
My question is, why does one tutorial use a preprocessing model provided, while the other actually implements python code? Is there a difference between the two tutorials that requires them to use their specific preprocessing methods?
Thank you!
I'm interested in NLP and I come up with Tensorflow and Bert, both seem to be from Google and both seem to be the best thing for Sentiment Analysis as of today but I don't understand what are they exactly and what is the difference between them... Can someone explain?
Tensorflow is an open-source library for machine learning that will let you build a deep learning model/architecture. But the BERT is one of the architectures itself. You can build many models using TensorFlow including RNN, LSTM, and even the BERT. The transformers like the BERT are a good choice if you just want to deploy a model on your data and you don't care about the deep learning field itself. For this purpose, I recommended the HuggingFace library that provides a straightforward way to employ a transformer model in just a few lines of code. But if you want to take a deeper look at these models, I will suggest you to learns about the well-known deep learning architectures for text data like RNN, LSTM, CNN, etc., and try to implement them using an ML library like Tensorflow or PyTorch.
Bert and Tensorflow is not different thing , There are not only 2, but many implementations of BERT. Most are basically equivalent.
The implementations that you mentioned are:
The original code by Google, in Tensorflow. https://github.com/google-research/bert
Implementation by Huggingface, in Pytorch and Tensorflow, that reproduces the same results as the original implementation and uses the same checkpoints as the original BERT article. https://github.com/huggingface/transformers
These are the differences regarding different aspects:
In terms of results, there is no difference in using one or the other, as they both use the same checkpoints (same weights) and their results have been checked to be equal.
In terms of reusability, HuggingFace library is probably more reusable, as it is designed specifically for that. Also, it gives you the freedom of choosing TensorFlow or Pytorch as deep learning framework.
In terms of performance, they should be the same.
In terms of community support (e.g. asking questions in github or stackoverflow about them), HuggingFace library is better suited, as there are a lot of people using it.
Apart from BERT, the transformers library by HuggingFace has implementations for lots of models: OpenAI GPT-2, RoBERTa, ELECTRA, ...
I would like to use AllenNLP Interpret (code + demo) with a PyTorch classification model trained with HuggingFace (electra base discriminator). Yet, it is not obvious to me, how I can convert my model, and use it in a local allen-nlp demo server.
How should I proceed ?
Thanks in advance
If your task is binary classification, you can look at the BoolQ example in https://github.com/allenai/allennlp-models/blob/main/training_config/classification/boolq_roberta.jsonnet. You can change that configuration to use a different model (such as Electra).
We also just put some new documentation out for the Interpret functionality: https://guide.allennlp.org/interpret
To give you a more specific answer, I'll need to know some more details, like what the task is you're trying to solve, how you trained the original model, etc.
I was going through the tutorial of speech emotion recognition and in between saw an "MLPClassifier(Multilayer_perceptron)" which was imported from the sklearn. And there are lots of other like Random forest and linear Regression, standardscalar, GridSearchCV, etc. I was searching for tutorials or steps to how can I create these types of classifiers or modules on my own?
When I searched regarding these, I was getting examples of tutorials of the use cases of predefined classifiers of sklearn and third party claassifiers. Like above specified.
If you guys know any tutorial or steps to achieve these please suggest to me.
Fro MLP, The implementation is quite easy, there is good explanation on how to implement on the Coursera's ML introdcution look to week 4 and week 5, for Linear and logistic regression look to week 2 and week 3. Look at this link for implementing CART, random forests are quite similar I think you can figure out how to implement them easily if you are able to implement CART. For SVM and kernel methods you can look to this repo
Please suggest a good machine learning classifier for truecasing of dataset.
Also, Is it possible to specify out own rules/features for truecasing in such a classifier? Thanks for all your suggestions.
Thanks
I implemented a version of a truecaser in Python. It can be trained for any language when you provide enough data (i.e. correctly cased sentences).
For English, it achieves an accuracy of 98.38% on sample sentences from Wikipedia. A pre-trained model for English is provided.
You can find it here:
https://github.com/nreimers/truecaser
Please take a look at this whitepaper.
http://www.cs.cmu.edu/~llita/papers/lita.truecasing-acl2003.pdf
They report 98% of accuracy.