I am looking to optimize some of the sentence transformer models from huggingface using optimum library. I am following the below documentation:
https://huggingface.co/blog/optimum-inference
I understand the process but I am not able to use model_id because our network restricts accessing huggingface using its APIs. I have downloaded these models locally and I am trying to following the same procedure but I cannot find any examples on how to pass the onnx checkpoint model to ORTOptimizer class.
This is how I converted the vanilla model to onnx checkpoint:
from pathlib import Path
import transformers
from transformers.onnx import FeaturesManager
from transformers import AutoConfig, AutoTokenizer, AutoModel
# load model and tokenizer
feature = "question-answering"
model_path=r'local\path\to\all-MiniLM-L12-v2'
model = AutoModel.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path)
model_kind, model_onnx_config = FeaturesManager.check_supported_model_or_raise(model, feature=feature)
onnx_config = model_onnx_config(model.config)
onnx_path = Path("onnx")
# export
onnx_inputs, onnx_outputs = transformers.onnx.export(
preprocessor=tokenizer,
model=model,
config=onnx_config,
opset=13,
output=onnx_path
)
This saves the model as onnx checkpoint. Then I try to load the onnx model along with its original model configuration:
from optimum.onnxruntime import ORTModelForQuestionAnswering
task = "question-answering"
ORTModelForQuestionAnswering(model= onnx_path, config = r'local\path\to\all-MiniLM-L12-v2\config.json')
I get the following error from the above code:
C:\Anaconda3\lib\site-packages\optimum\onnxruntime\modeling_ort.py in __init__(self, model, config, use_io_binding, **kwargs)
641
642 def __init__(self, model=None, config=None, use_io_binding=True, **kwargs):
--> 643 super().__init__(model, config, use_io_binding, **kwargs)
644 # create {name:idx} dict for model outputs
645 self.model_outputs = {output_key.name: idx for idx, output_key in enumerate(self.model.get_outputs())}
C:\Anaconda3\lib\site-packages\optimum\onnxruntime\modeling_ort.py in __init__(self, model, config, use_io_binding, **kwargs)
123 self.model_save_dir = kwargs.get("model_save_dir", None)
124 self.latest_model_name = kwargs.get("latest_model_name", "model.onnx")
--> 125 self.providers = model.get_providers()
126 self._device = get_device_for_provider(self.providers[0])
127
AttributeError: 'WindowsPath' object has no attribute 'get_providers'
How do I resolve this issue and is there any examples for the process that I am trying to do i.e. load sentence transformer models locally rather than using model_id.
Appreciate all the help!
Related
I am trying to train a model for NMT on a custom dataset. I found this great tutorial on youtube along with the accompanying repo, but it uses an old version of PyTorch and torchtext. More recent versions of torchtext have removed the Field and BucketIterator classes.
I looked for more recent tutorials. The closest thing I could find was this medium post (again with the accompanying code) which worked with a custom dataset for text classification. I tried to replicate the code with my problem and got this far:
from os import PathLike
from torch.utils.data import Dataset
from torchtext.vocab import Vocab
import pandas as pd
from .create_vocab import tokenizer
class ParallelCorpus(Dataset):
"""A parallel corpus for training a machine translation model"""
def __init__(self,
corpus_path: str | PathLike,
source_vocab: Vocab,
target_vocab: Vocab
):
super().__init__()
self.corpus = pd.read_csv(corpus_path)
self.source_vocab = source_vocab
self.target_vocab = target_vocab
def __len__(self):
return len(self.corpus)
def __getitem__(self, index: int):
source_sentence = self.corpus.iloc[index, 0]
source = [self.source_vocab["<sos>"]]
source.extend(
self.source_vocab.lookup_indices(tokenizer(source_sentence))
)
source.append(self.source_vocab["<eos>"])
target_sentence = self.corpus.iloc[index, 1]
target = [self.target_vocab["<sos>"]]
target.extend(
self.target_vocab.lookup_indices(tokenizer(target_sentence))
)
target.append(self.target_vocab["<eos>"])
return source, target
My question is: is this the correct way to implement parallel corpora for pytorch? And where can I find more information about this since the documentation wasn't much help.
Thank you in advance and sorry if this is against the rules.
When running the BERT NLP Model of HuggingFace, it ouputs the OSError below. Below is the code and Error,
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
import requests
from bs4 import BeautifulSoup
import re
## Instantiate Model
# the nlp bert pre-trained model used here is from website huggingface. the website is " https://huggingface.co/nlptown/bert-base-multilingual-uncased-sentiment "
tokenizer = AutoTokenizer.from_pretrained('nlptown/bert-base-multilingual-uncased-sentiment')
model = AutoModelForSequenceClassification.from_pretrained('nlptown/bert-base-multilingual-uncased-sentiment')
## Encode and Calculate Sentiment ( Now entering string words to test the sentiment score )
tokens = tokenizer.encode('I hated this, absolutely the worst', return_tensors='pt' )
result = model(tokens)
print(result)
Error Below,
OSError: Can't load the model for
'nlptown/bert-base-multilingual-uncased-sentiment'. If you were trying
to load it from 'https://huggingface.co/models', make sure you don't
have a local directory with the same name. Otherwise, make sure
'nlptown/bert-base-multilingual-uncased-sentiment' is the correct path
to a directory containing a file named pytorch_model.bin, tf_model.h5,
model.ckpt or flax_model.msgpack.
I get this error when i try to load the model into keras as pb file. This is how file structure looks like
-SavedModel
-variables
variables.index
saved_model.pb
-SavedImgModel
-variables
variables.index
saved_model.pb
def load_model():
global model
global img_model
model_path = os.path.join('.','SavedModel\\',)
img_modelpath = os.path.join('.','SavedImgModel\\')
print(model_path)
print(("* Loading Keras model and Flask starting server...please wait until server has fully started"))
model = tf.keras.models.load_model(model_path)
img_model = tf.keras.models.load_model(img_modelpath)
# initialize our Flask application and the Keras model
app = Flask(__name__)
load_model()
detector = MTCNN(
Error as :
raise errors_impl.OpError(None, None, error_message, errors_impl.UNKNOWN)
tensorflow.python.framework.errors_impl.OpError: NewRandomAccessFile failed to Create/Open: .\SavedModel\variables\variables.data-00000-of-00001 : The system cannot find the file specified.
; No such file or director
I have trained transformer using simpletransformers model on colab ,downloaded the searialized model and i have little issues on using it to make inferences.
Loading the model on model on jupyter works but while using it with fastapi gives an error
This is how I,m using it on jupyter:
from scipy.special import softmax
label_cols = ['art', 'politics', 'health', 'tourism']
model = torch.load("model.bin")
pred = model.predict(['i love politics'])[1]
preds = softmax(pred,axis=1)
preds
It gives the following result:array([[0.00230123, 0.97465035, 0.00475409, 0.01829433]])
I have tried using fastapi as follows but keeps on getting an error:
from pydantic import BaseModel
class Message(BaseModel):
text : str
model = torch.load("model.bin")
#app.post("/predict")
def predict_health(data: Message):
prediction = model.predict(data.text)[1]
preds = softmax(prediction, axis=1)
return {"results": preds}
Could you please specify the error you get, otherwise its quite hard to see debug
Also it seems that the model.predict function in the jupyter code gets an array as input, while in your fastapi code you are passing a string directly to that function.
So maybe try
...
prediction = model.predict([data.text])[1]
...
It's hard to say without the error.
In case it helps you could have a look at this article that shows how to build classification with Hugging Face transformers (Bart Large MNLI model) and FastAPI: https://nlpcloud.io/nlp-machine-learning-classification-api-production-fastapi-transformers-nlpcloud.html
I am trying to use mlflow to save a model and then load it later to make predictions.
I'm using a xgboost.XGBRegressor model and its sklearn functions .predict() and .predict_proba() to make predictions but it turns out that mlflow doesn't support models that implements the sklearn API, so when loading the model later from mlflow, mlflow returns an instance of xgboost.Booster, and it doesn't implements the .predict() or .predict_proba() functions.
Is there a way to convert a xgboost.Booster back into a xgboost.sklearn.XGBRegressor object that implements the sklearn API functions?
Have you tried wrapping up your model in custom class, logging and loading it using mlflow.pyfunc.PythonModel?
I put up a simple example and upon loading back the model it correctly shows <class 'xgboost.sklearn.XGBRegressor'> as a type.
Example:
import xgboost as xgb
xg_reg = xgb.XGBRegressor(...)
class CustomModel(mlflow.pyfunc.PythonModel):
def __init__(self, xgbRegressor):
self.xgbRegressor = xgbRegressor
def predict(self, context, input_data):
print(type(self.xgbRegressor))
return self.xgbRegressor.predict(input_data)
# Log model to local directory
with mlflow.start_run():
custom_model = CustomModel(xg_reg)
mlflow.pyfunc.log_model("custome_model", python_model=custom_model)
# Load model back
from mlflow.pyfunc import load_model
model = load_model("/mlruns/0/../artifacts/custome_model")
model.predict(X_test)
Output:
<class 'xgboost.sklearn.XGBRegressor'>
[ 9.107417 ]
I have a xgboost.core.Booster object and it can make return probability calculations as follows your_Booster_model_object.predict(your_xgboost_dmatrix_dataset).