I am trying to save and load with the following code but it is not working. It is showing me an error telling me that it is not able to find the model. Am I missing something? I'm using Google Colab. Thank you
import keras
callbacks_list = [keras.callbacks.EarlyStopping(monitor='val_loss',patience=6,),
keras.callbacks.ModelCheckpoint(filepath='my_model.h5',monitor='val_loss',mode='min', save_freq='epoch',save_best_only=True,)]
model.compile(loss=MeanAbsoluteError(), optimizer='Adam',metrics=[RootMeanSquaredError()])
history= model.fit(X_train, y_train,batch_size=512, epochs=100,callbacks=callbacks_list,validation_data=(X_val, y_val))
from tensorflow.keras.models import load_model
#save model to single file
model.save('my_model.h5')
#To load model
model = load_model('my_model.h5')
Since you are using Google Colab, you must mount your drive to access the data on Colab. Assuming that the notebook you are executing is in the directory my_dir (update the path according to YOUR particular path) you can add the following code to a cell before your save and load code:
from google.colab import drive
drive.mount('/content/drive') # mounts the drive
%cd /content/drive/MyDrive/my_dir/ # moves your position inside the directory where you are executing the code
# ... your code to save and your code to load
I'm new to torch audio and i'm following the this tutorial step by step. I'm having a problem loading an mp3 audio using torchaudio.info(path).
Here is my code:
metadata = torchaudio.info(SAMPLE_MP3_PATH)
print(metadata)
Here is the error that i'm getting:
..
RuntimeError: Error opening '_sample_data\\steam.mp3': File contains data in an unknown format.
torch: v1.9.1+cpu
torchaudio: v0.9.1
torchaudio.info will call its backend to show the information.
If you use it in windows, and the backend is Soundfile, then this problem will occur. Because Soundfile does not support mp3 format.
You can use the below code to see the available formats.
import soundfile as sf
sf.available_formats()
I am trying to read all the frames of a video in Opencv, but after a certain point it is giving me "None" as a result, due to which I am not able to perform the processing tasks which I plan to do.
import time
import numpy as np
import cv2
# Create our body classifier
body_classifier = cv2.CascadeClassifier('dataset/haarcascade_fullbody.xml')
# Initiate video capture for video file
cap = cv2.VideoCapture('dataset/ped.mp4')
# Loop once video is successfully loaded
while (cap.isOpened()):
time.sleep(0.05)
# Read first frame
ret, frame = cap.read()
print(frame)
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
The error which I am getting is here below:
enter image description here
Some points I would like to clarify before is that:
The path for the video mentioned is correct and also I have tried a
changing the "time.sleep" and the speed of the video but nothing worked and got the same error again.
Can anyone please tell what is the reason behind the "None" value and how could this be resolved.
I am adding the link of the video below:
https://drive.google.com/file/d/1HtNrm5rI9rtMJRqoSrqc5d1EFmTX5IXS/view?usp=sharing
I downloaded Fashion MNIST dataset from kaggle using dowload_data() function in fastai library.
downloaded_data = download_data("https://www.kaggle.com/zalando-research/fashionmnist/download")
output -
PosixPath('/root/.fastai/data/download.tgz')
download_data saves it as .tgz file, now I use untar_data().
path = untar_data('/root/.fastai/data/download.tgz')
output -
PosixPath('/root/.fastai/data/download.tgz')
Which did not extract .tgz file. How do I use this dataset in fastai library?
In fastai library, the download_data gives you a pathlib.PosixPath file, not the exact file, you need to use another unzipping library to extract the data.
If you just need the MNIST data from fast ai, here's an easier way:
from fastai import datasets
import gzip, pickle
MNIST_URL='http://deeplearning.net/data/mnist/mnist.pkl'
path = datasets.download_data(MNIST_URL, ext='.gz')
with gzip.open(path, 'rb') as f:
((x_train, y_train), (x_valid, y_valid), _) = pickle.load(f, encoding='latin-1')
I am attempting to retrain inception v3 on a new image set.
When I try to save the model I receive an error.
I have tried:
tf.keras.models.save_model(model, filename)
and
model.save(filename)
and
tf.contrib.saved_model.save_keras_model(model, filename)
All give me a similar error, Module has no 'name'
I have attached my code relevant to the problem.
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import argparse
import os
import sys
import numpy as np
import tensorflow as tf
import tensorflow_hub as hub
import matplotlib.pyplot as plt
FLAGS = None
def create_model(m, img_data):
# load feature extractor (inception_v3)
features_extractor_layer = tf.keras.layers.Lambda(m, input_shape=img_data.image_shape)
# make pre-trained layers un-trainable
features_extractor_layer.trainable = False
print(features_extractor_layer.name)
# add new activation layer to train to our classes
model = tf.keras.Sequential([
features_extractor_layer,
tf.keras.layers.Dense(img_data.num_classes, activation='softmax')
])
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
return model
def get_and_gen_images(module):
"""
get images from image directory or url
:param module: module (to get required image size info
:return: batched image data
"""
data_name = os.path.splitext(os.path.basename(FLAGS.image_dir_or_url))[0]
print("data: ", data_name)
# download images to cache if not already
if FLAGS.image_dir_or_url.startswith('https://'):
data_root = tf.keras.utils.get_file(data_name,
FLAGS.image_dir_or_url,
untar=True,
cache_dir=os.getcwd())
else: # specify directory with images
data_root = tf.keras.utils.get_file(data_name,
FLAGS.image_dir_or_url)
# get image size for specific module
image_size = hub.get_expected_image_size(module)
# TODO: this is where to add noise, rotations, shifts, etc.
image_generator = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1. / 255, validation_split=0.2)
# create image stream
train_image_data = image_generator.flow_from_directory(str(data_root),
target_size=image_size,
batch_size=FLAGS.batch_size,
subset='training')
validation_image_data = image_generator.flow_from_directory(str(data_root),
target_size=image_size,
batch_size=FLAGS.batch_size,
subset='validation')
return train_image_data, validation_image_data
# load module (will download from url or directory_
module = hub.Module(FLAGS.tfhub_module)
# generate image stream
train_image_data, validation_image_data = get_and_gen_images(module)
model = create_model(module, train_image_data)
model.summary()
file = FLAGS.saved_model_dir + "/modelname.h5"
model.save(file)
This should save a ".h5" model file, but I receive a naming error:
Traceback (most recent call last):
File "/home/raphy/projects/vmi/tf_cpu/retrain.py", line 305, in <module>
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "/home/raphy/.local/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "/home/raphy/projects/vmi/tf_cpu/retrain.py", line 205, in main
model.save(file)
File "/home/raphy/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/sequential.py", line 319, in save
save_model(self, filepath, overwrite, include_optimizer)
File "/home/raphy/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/saving.py", line 105, in save_model
'config': model.get_config()
File "/home/raphy/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/sequential.py", line 326, in get_config
'config': layer.get_config()
File "/home/raphy/.local/lib/python3.6/site-packages/tensorflow/python/keras/layers/core.py", line 756, in get_config
function = self.function.__name__
AttributeError: 'Module' object has no attribute '__name__'
I want to save the model in the format of the tf_hub models.
As specified in Hosting TF Hub TF Org Link,
If you are interested in hosting your own repository of models that
are loadable with the tensorflow_hub library, your HTTP distribution
service should follow the following protocol.
In other words, you cannot load any Model using TF Hub, but you can load only the Modules present in the TF Hub Modules Site.
If you want to Load your Saved Model, you can do it using tf.saved_model.load.
But if you want to do it using TF Hub, please refer this link.
Also, mentioning the instructions below just in case if the link doesn't work:
Hosting your own models:
TensorFlow Hub provides an open repository of trained models at thub.dev. The tensorflow_hub library can load models from this repository and other HTTP based repositories of machine learning models. In particular the protocol allows to use the URL identifying the model both for the documentation of the model and the endpoint to fetch the model.
If you are interested in hosting your own repository of models that are loadable with the tensorflow_hub library, your HTTP distribution service should follow the following protocol.
Protocol:
When a URL such as https://example.com/model is used to identify a model to load or instantiate, the model resolver will attempt to download a compressed tarball from the URL after appending a query parameter ?tf-hub-format=compressed.
The query param is to be interpreted as a comma separated list of the model formats that the client is interested in. For now only the "compressed" format is defined.
The compressed format indicates that the client expects a tar.gz archive with the model contents. The root of the archive is the root of the model directory and should contain a SavedModel, as in this example:
# Create a compressed model from a SavedModel directory.
$ tar -cz -f model.tar.gz --owner=0 --group=0 -C /tmp/export-model/ .
# Inspect files inside a compressed model
$ tar -tf model.tar.gz
./
./variables/
./variables/variables.data-00000-of-00001
./variables/variables.index
./assets/
./saved_model.pb
Tarballs for use with the deprecated hub.Module() API from TF1 will also contain a ./tfhub_module.pb file. The hub.load() API for TF2 SavedModels ignores such a file.
The tensorflow_hub library expects that model URLs are versioned and that the model content of a given version is immutable, so that it can be cached indefinitely.