I have trained transformer using simpletransformers model on colab ,downloaded the searialized model and i have little issues on using it to make inferences.
Loading the model on model on jupyter works but while using it with fastapi gives an error
This is how I,m using it on jupyter:
from scipy.special import softmax
label_cols = ['art', 'politics', 'health', 'tourism']
model = torch.load("model.bin")
pred = model.predict(['i love politics'])[1]
preds = softmax(pred,axis=1)
preds
It gives the following result:array([[0.00230123, 0.97465035, 0.00475409, 0.01829433]])
I have tried using fastapi as follows but keeps on getting an error:
from pydantic import BaseModel
class Message(BaseModel):
text : str
model = torch.load("model.bin")
#app.post("/predict")
def predict_health(data: Message):
prediction = model.predict(data.text)[1]
preds = softmax(prediction, axis=1)
return {"results": preds}
Could you please specify the error you get, otherwise its quite hard to see debug
Also it seems that the model.predict function in the jupyter code gets an array as input, while in your fastapi code you are passing a string directly to that function.
So maybe try
...
prediction = model.predict([data.text])[1]
...
It's hard to say without the error.
In case it helps you could have a look at this article that shows how to build classification with Hugging Face transformers (Bart Large MNLI model) and FastAPI: https://nlpcloud.io/nlp-machine-learning-classification-api-production-fastapi-transformers-nlpcloud.html
Related
When I try to run my code for Donut for DocVQA model, I got the following error
"""Test"""
from donut import DonutModel
from PIL import Image
import torch
model = DonutModel.from_pretrained(
"naver-clova-ix/donut-base-finetuned-cord-v2")
if torch.cuda.is_available():
model.half()
device = torch.device("cuda")
model.to(device)
else:
model.encoder.to(torch.bfloat16)
model.eval()
image = Image.open(
"./src/png-report-page_capone_v1_August_2017_GMS_Balance_Sheet_0.png").convert("RGB")
output = model.inference(image=image, prompt="<s_cord-v2>")
print(output)
The error`
ValueError: The following `model_kwargs` are not used by the model: ['encoder_outputs'] (note: typos in the generate arguments will also show up in this list)
Check the transformers library version.
For me, I reinstalled the 4.21.3 version and it worked.
I am trying to use mlflow to save a model and then load it later to make predictions.
I'm using a xgboost.XGBRegressor model and its sklearn functions .predict() and .predict_proba() to make predictions but it turns out that mlflow doesn't support models that implements the sklearn API, so when loading the model later from mlflow, mlflow returns an instance of xgboost.Booster, and it doesn't implements the .predict() or .predict_proba() functions.
Is there a way to convert a xgboost.Booster back into a xgboost.sklearn.XGBRegressor object that implements the sklearn API functions?
Have you tried wrapping up your model in custom class, logging and loading it using mlflow.pyfunc.PythonModel?
I put up a simple example and upon loading back the model it correctly shows <class 'xgboost.sklearn.XGBRegressor'> as a type.
Example:
import xgboost as xgb
xg_reg = xgb.XGBRegressor(...)
class CustomModel(mlflow.pyfunc.PythonModel):
def __init__(self, xgbRegressor):
self.xgbRegressor = xgbRegressor
def predict(self, context, input_data):
print(type(self.xgbRegressor))
return self.xgbRegressor.predict(input_data)
# Log model to local directory
with mlflow.start_run():
custom_model = CustomModel(xg_reg)
mlflow.pyfunc.log_model("custome_model", python_model=custom_model)
# Load model back
from mlflow.pyfunc import load_model
model = load_model("/mlruns/0/../artifacts/custome_model")
model.predict(X_test)
Output:
<class 'xgboost.sklearn.XGBRegressor'>
[ 9.107417 ]
I have a xgboost.core.Booster object and it can make return probability calculations as follows your_Booster_model_object.predict(your_xgboost_dmatrix_dataset).
i am trying to train a pytorch model on colab then save the model parameters and load it on my local computer.
After training, the model parameters are stored as below:
torch.save(Model.state_dict(),PATH)
loaded as below:
device = torch.device('cpu')
Model.load_state_dict(torch.load(PATH, map_location=device))
error:
AttributeError: 'Sequential' object has no attribute 'copy'
Does anyone know how to solve this issue?
Your question does not provide sufficient details to be answered correctly. If you are trying to save and load your own model and have a class definition for it see this well known answer and clarify why that's not sufficient for your use.
If you are loading a torch.nn.Sequential model then as far as I know simply loading the model directly and just using it should be sufficient. If it's not post on the pytorch forum what error you get.
For now look at my example show casing loading a sequential model and then using it without error:
# test for saving everything with torch.save
import torch
import torch.nn as nn
from pathlib import Path
from collections import OrderedDict
import numpy as np
import pickle
path = Path('~/data/tmp/').expanduser()
path.mkdir(parents=True, exist_ok=True)
num_samples = 3
Din, Dout = 1, 1
lb, ub = -1, 1
x = torch.torch.distributions.Uniform(low=lb, high=ub).sample((num_samples, Din))
f = nn.Sequential(OrderedDict([
('f1', nn.Linear(Din,Dout)),
('out', nn.SELU())
]))
y = f(x)
# save data torch to numpy
x_np, y_np = x.detach().cpu().numpy(), y.detach().cpu().numpy()
db2 = {'f': f, 'x': x_np, 'y': y_np}
torch.save(db2, path / 'db_f_x_y')
db3 = torch.load(path / 'db_f_x_y')
f3 = db3['f']
x3 = db3['x']
y3 = db3['y']
xx = torch.tensor(x3)
yy3 = f3(xx)
print(yy3)
there should be an official answer how to save and load nn.Sequential models How does one save torch.nn.Sequential models in pytorch properly? but for now torch.save and torch.load seem to work just fine.
The documentation and a few related post lead me to believe I should be able to get the feature names from the labels scikit-learn's LabelBinarizer is one-hot encoding.
I have a pipeline defined as following:
cat_pipeline = Pipeline([
('selector', DataFrameSelector(cat_attribs)),
('label_binarizer', LabelBinarizer()),
])
This works just fine, (note that DataFrameSelector is a custom class), however, it seems like I could extract the feature names like this:
feature_names = cat_pipeline.named_steps['label_binarizer'].get_feature_names()
I also tried substituting get_feature_names() with get_support() to not avail.
This is possible when using LabelEncoder on its own outside of a pipeline before one hot encoding as follows:
from sklearn.preprocessing import LabelEncoder
encoder = LabelEncoder()
housing_cat = housing["ocean_proximity"]
housing_cat_encoded = encoder.fit_transform(housing_cat)
housing_cat_encoded
print(encoder.classes_)
For further context please see the notebook I am working through:
https://github.com/ageron/handson-ml/blob/master/02_end_to_end_machine_learning_project.ipynb
I want to use my Tensorflow algorithm in an Android app. The Tensorflow Android example starts by downloading a GraphDef that contains the model definition and weights (in a *.pb file). Now this should be from my Scikit Flow algorithm (part of Tensorflow).
At the first glance it seems easy you just have to say classifier.save('model/') but the files saved to that folder are not *.ckpt, *.def and certainly not *.pb. Instead you have to deal with a *.pbtxt and a checkpoint (without ending) file.
I'm stuck there since quite a while. Here a code example to export something:
#imports
import tensorflow as tf
import tensorflow.contrib.learn as skflow
import tensorflow.contrib.learn.python.learn as learn
from sklearn import datasets, metrics
#skflow example
iris = datasets.load_iris()
feature_columns = learn.infer_real_valued_columns_from_input(iris.data)
classifier = learn.LinearClassifier(n_classes=3, feature_columns=feature_columns,model_dir="modeltest")
classifier.fit(iris.data, iris.target, steps=200, batch_size=32)
iris_predictions = list(classifier.predict(iris.data, as_iterable=True))
score = metrics.accuracy_score(iris.target, iris_predictions)
print("Accuracy: %f" % score)
The files you get are:
checkpoint
graph.pbtxt
model.ckpt-1.meta
model.ckpt-1-00000-of-00001
model.ckpt-200.meta
model.ckpt-200-00000-of-00001
Many possible workarounds I found would require having the GraphDef in a variable (don't know how with Scikit Flow). Or a Tensorflow session which doesn't seem to be required using Scikit Flow.
To save as pb file, you need to extract the graph_def from the constructed graph. You can do that as--
from tensorflow.python.framework import tensor_shape, graph_util
from tensorflow.python.platform import gfile
sess = tf.Session()
final_tensor_name = 'results:0' #Replace final_tensor_name with name of the final tensor in your graph
#########Build your graph and train########
## Your tensorflow code to build the graph
###########################################
outpt_filename = 'output_graph.pb'
output_graph_def = sess.graph.as_graph_def()
with gfile.FastGFile(outpt_filename, 'wb') as f:
f.write(output_graph_def.SerializeToString())
If you want to convert your trained variables to constants (to avoid using ckpt files to load the weights), you can use:
output_graph_def = graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), [final_tensor_name])
Hope this helps!