Is it possible to convert the PyTorch Model to ONNX without exporting and further use it as an ONNX object directly in the script.
You can export to memory, something like this:
import io
f = io.BytesIO()
torch.onnx.export(model, sample_inputs, f, ...)
onnx_model = onnx.load_model_from_string(f.getvalue())
Related
I have trained transformer using simpletransformers model on colab ,downloaded the searialized model and i have little issues on using it to make inferences.
Loading the model on model on jupyter works but while using it with fastapi gives an error
This is how I,m using it on jupyter:
from scipy.special import softmax
label_cols = ['art', 'politics', 'health', 'tourism']
model = torch.load("model.bin")
pred = model.predict(['i love politics'])[1]
preds = softmax(pred,axis=1)
preds
It gives the following result:array([[0.00230123, 0.97465035, 0.00475409, 0.01829433]])
I have tried using fastapi as follows but keeps on getting an error:
from pydantic import BaseModel
class Message(BaseModel):
text : str
model = torch.load("model.bin")
#app.post("/predict")
def predict_health(data: Message):
prediction = model.predict(data.text)[1]
preds = softmax(prediction, axis=1)
return {"results": preds}
Could you please specify the error you get, otherwise its quite hard to see debug
Also it seems that the model.predict function in the jupyter code gets an array as input, while in your fastapi code you are passing a string directly to that function.
So maybe try
...
prediction = model.predict([data.text])[1]
...
It's hard to say without the error.
In case it helps you could have a look at this article that shows how to build classification with Hugging Face transformers (Bart Large MNLI model) and FastAPI: https://nlpcloud.io/nlp-machine-learning-classification-api-production-fastapi-transformers-nlpcloud.html
I am trying to use mlflow to save a model and then load it later to make predictions.
I'm using a xgboost.XGBRegressor model and its sklearn functions .predict() and .predict_proba() to make predictions but it turns out that mlflow doesn't support models that implements the sklearn API, so when loading the model later from mlflow, mlflow returns an instance of xgboost.Booster, and it doesn't implements the .predict() or .predict_proba() functions.
Is there a way to convert a xgboost.Booster back into a xgboost.sklearn.XGBRegressor object that implements the sklearn API functions?
Have you tried wrapping up your model in custom class, logging and loading it using mlflow.pyfunc.PythonModel?
I put up a simple example and upon loading back the model it correctly shows <class 'xgboost.sklearn.XGBRegressor'> as a type.
Example:
import xgboost as xgb
xg_reg = xgb.XGBRegressor(...)
class CustomModel(mlflow.pyfunc.PythonModel):
def __init__(self, xgbRegressor):
self.xgbRegressor = xgbRegressor
def predict(self, context, input_data):
print(type(self.xgbRegressor))
return self.xgbRegressor.predict(input_data)
# Log model to local directory
with mlflow.start_run():
custom_model = CustomModel(xg_reg)
mlflow.pyfunc.log_model("custome_model", python_model=custom_model)
# Load model back
from mlflow.pyfunc import load_model
model = load_model("/mlruns/0/../artifacts/custome_model")
model.predict(X_test)
Output:
<class 'xgboost.sklearn.XGBRegressor'>
[ 9.107417 ]
I have a xgboost.core.Booster object and it can make return probability calculations as follows your_Booster_model_object.predict(your_xgboost_dmatrix_dataset).
I want to check what is the default data type expected in the input layer, and how to change it?
If you have keras model just do:
print(model.inputs[0].dtype
You can pass desired datatype while creating a model. For example:
import tensorflow as tf
print(tf.__version__) # 2.0.1
x = tf.keras.layers.Input((2,), dtype=tf.float64)
res = tf.keras.layers.Dense(2, dtype=tf.float64)(x)
model = tf.keras.models.Model(x, res)
print(model.inputs[0].dtype, model.outputs[0].dtype) # <dtype: 'float64'> <dtype: 'float64'>
Is it possible to use Keras model objects with CalibratedClassifierCV from sklearn.calibration? Or is there another way to performa isotonic regression in sklearn/other python packages without having to pass it a model object.
I tried using the sklearn wrapper for Keras, but it didn't work. Here is the doc for the CalibratedClassifierCV class.
You can train an isotonic regression a posteriori, after prediction. Let 'file1' be a csv containing your predictions pred and real observed events obs on a subset of data. Ideally, this subset has never been used before (not even in Keras training). Let file2 contain the predictions you want to calibrate (Keras predictions for the test set).
import pandas as pd
from sklearn.isotonic import IsotonicRegression
never_seen=pd.read_csv('file1')
uncalibrated=pd.read_csv('file2')
ir = IsotonicRegression( out_of_bounds = 'clip' )
ir.fit( never_seen.pred,never_seen.obs )
p_calibrated = ir.transform( uncalibrated.pred )
I am trying to convert a RandomForestClassifier model of the MNIST dataset into a CoreML model using coremltools.
I use fetch_mldata from sklearn.datasets to import the data; my samples (data) are stored in a Numpy array X, and the labels (target) are stored in a Numpy array Y.
My model is generated like this:
from sklearn.ensemble import RandomForestClassifier
rfc = RandomForestClassifier()
rfc.fit(X, Y)
I then try to convert the model:
import coremltools
model = coremltools.converters.sklearn.convert(rfc)
The CoreML model input then looks like this:
input {
name: "input"
type {
multiArrayType {
shape: 784
dataType: DOUBLE
}
}
}
The problem is the multiArrayType. It is much easier to deal with a pixel-buffer in iOS, so various sources point to this syntax (however, they use Caffe rather than sklearn):
model = coremltools.converters.sklearn.convert(rfc, image_input_names='data')
However, this gives me an error message:
TypeError: convert() got an unexpected keyword argument 'image_input_names'
I have tried to find the documentation for these parameters, but I have only found some few example using Caffe, and they do not seem to get this error.