Is there a way to save a Keras model (using pickle, joblib, model.to_json() or tf.keras.models.save_model) that has been incorporated into an sklearn or imblearn pipeline via a scikit-learn wrapper such as KerasClassifier?
Related
I know we can train sequential model in keras by estimator api, use keras_to_estimator, however it doesn't work in keras subclass model.
I have an EfficientNet model (tensorflow.keras==2.4) and would like to use innvestigate to inspect the results, but it requires keras==2.2.4
Training code:
tensorflow.keras.__version__ # 2.4
model = tf.keras.applications.EfficientNetB1(**params)
# do training
model.save('testModel')
I have the model saved as file but can not load it into Keras 2.2.4. This is the point where I'm stuck, I couldn't figure out what to do to convert the model.
Use Innvestigate:
keras.__version__ # 2.2.4
keras.model.load_model('testModel') # Error
# some more stuff...
I also found this thread, might try it, but since efficient net has > 350 layers it is not really applicable
How to load tf.keras models with keras
I don't know if it's actually possible to convert models between tensorflow.keras and keras, I appreciate all help I can get.
Due to version incompatibility between tensorflow as keras, you were not able to load model.
Your issue will be resolved, once you upgrade keras and tensorflow to 2.5.
If I have a trained model in Using pickle, or Joblib.
Lets say its Logistic regression or XGBoost.
I would like to host that model in AWS Sagemaker as endpoint without running a training job.
How to achieve that.
#Lets Say myBucketName contains model.pkl
model = joblib.load('filename.pkl')
# X_test = Numpy Array
model.predict(X_test)
I am not interested to sklearn_estimator.fit('S3 Train, S3 Validate' ) , I have the trained model
For Scikit Learn for example, you can get inspiration from this public demo https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python-sdk/scikit_learn_randomforest/Sklearn_on_SageMaker_end2end.ipynb
Step 1: Save your artifact (eg the joblib) compressed in S3 at s3://<your path>/model.tar.gz
Step 2: Create an inference script with the deserialization function model_fn. (Note that you could also add custom inference functions input_fn, predict_fn, output_fn but for scikit the defaults function work fine)
%%writefile inference_script.py. # Jupiter command to create file in case you're in Jupiter
import joblib
import os
def model_fn(model_dir):
clf = joblib.load(os.path.join(model_dir, "model.joblib"))
return clf
Step 3: Create a model associating the artifact with the right container
from sagemaker.sklearn.model import SKLearnModel
model = SKLearnModel(
model_data='s3://<your path>/model.tar.gz',
role='<your role>',
entry_point='inference_script.py',
framework_version='0.23-1')
Step 4: Deploy!
model.deploy(
instance_type='ml.c5.large', # choose the right instance type
initial_instance_count=1)
I want to convert a Keras model to Tensorflow Lite model. When I examined the documentation, it is stated that we can use tf.keras HDF5 models as input. Does it mean I can use my saved HDF5 Keras model as input to it or tf.keras HDF5 model and Keras HDF5 models are different things?
Documentation: https://www.tensorflow.org/lite/convert
Edit: I could convert my Keras model to Tensorflow Lite model with using this API, but I didn't test it yet. My code:
converter = tf.lite.TFLiteConverter.from_keras_model_file(path + 'plant-
recognition-model.h5')
tflite_model = converter.convert()
with open('plant-recognition-model.tflite', 'wb') as f:
f.write(tflite_model)
tf.keras HDF5 model and Keras HDF5 models are not different things, except for inevitable software version update synchronicity. This is what the official docs say:
tf.keras is TensorFlow's implementation of the Keras API specification. This is a high-level API to build and train models that includes first-class support for TensorFlow-specific functionality
If the convertor can convert a keras model to tf.lite, it will deliver same results. But tf.lite functionality is more limited than tf.keras. If this feature set is not enough for you, you can still work with tensorflow, and enjoy its other advantages.
May be, it won't take too long before your models can run on a smartphone.
I have trained a keras model and saved it to later make predictions. However, I loaded the saved model using:
from keras.models import load_model
#Restore saved keras model
restored_keras_model = load_model("C:/*******/saved_model.hdf5")
Now I would like to save an image of the loaded model so I can visualize it before using it before making predictions.
Is there a way of doing this in keras or is the use of another library required?
Yes, in addition to doing a restored_keras_model.summary(), you can save the model architecture as a png file using the plot_model API.
from keras.utils import plot_model
plot_model(restored_keras_model, to_file='model.png')
https://keras.io/visualization/#model-visualization