How to use ImageDataGenerator and predict_generator on a single JPEG file in Keras?
I am having a single jpeg and i want to predict the probability using model trained using model.fita-generator function.
If you just have a single .jpeg, you don't need to use the ImageDataGenerator. In the code below I'm assuming you trained your model with RGB images sized 150px x 150px.
img = image.load_img(img_path, target_size=(150, 150))
img_tensor = image.img_to_array(img)
img_tensor = np.expand_dims(img_tensor, axis=0)
img_tensor /= 255.
model.predict(img_tensor)
For more info, check out Francois Chollet's excellent Ipython Notebooks. Specifically, Line (In [2]) of https://github.com/fchollet/deep-learning-with-python-notebooks/blob/master/5.4-visualizing-what-convnets-learn.ipynb
In this section, he looks at the intermediate activation layers for an image that wasn't in his train_generator. He loads in a model he created in another Ipython notebook: https://github.com/fchollet/deep-learning-with-python-notebooks/blob/master/5.2-using-convnets-with-small-datasets.ipynb
Related
I'm trying to build a CNN model in order to classify an image, but whenever the training is done and I try to feed it a single image (from the training dataset) it misclassifies this image always.
Please take a look at the code I wrote below.
Thank you in advance.
First, I declared an Image Data Generator for both my training and testing sets:
train_datagen = ImageDataGenerator(rescale = 1./255, rotation_range=20, horizontal_flip = True,
validation_split=0.3)
test_datagen = ImageDataGenerator(rescale = 1./255,validation_split=0.3)
Then, I used the flow_from_directory() function to load the images:
train_generator = train_datagen.flow_from_directory(
data_dir,
shuffle=False,
subset='training',
target_size = (224, 224),
class_mode = 'categorical'
)
test_generator = test_datagen.flow_from_directory(
data_dir,
shuffle=False,
subset='validation',
target_size = (224, 224),
class_mode = 'categorical'
)
I then loaded a pretrained model and added a few layers to build my model:
pretrained_model = VGG16(weights="imagenet", include_top=False,
input_tensor=input_shape)
pretrained_model.trainable = False
model = tf.keras.Sequential([
pretrained_model,
Flatten(name="flatten"),
Dense(3, activation="softmax")
])
I then trained the model :
INIT_LR = 3e-4
EPOCHS = 15
opt = Adam(lr=INIT_LR)
model.compile(loss="categorical_crossentropy", optimizer='Adam', metrics=["accuracy"])
H = model.fit(
train_generator,
validation_data=test_generator,
epochs=EPOCHS,
verbose= 1)
Then came the part to predict a single image:
I chose an image that was part of the training set, I even overfitted the model to make sure the predictions should be correct, but it was giving me wrong results for every image I input to the model.
I tried the following ways:
image = image.load_img(url,target_size = (224, 224))
img = tf.keras.preprocessing.image.img_to_array(image)
img = np.array([img])
img = img.astype('float32') / 255.
img = tf.keras.applications.vgg16.preprocess_input(img)
This didn't work
image = cv2.imread(url)
image = cv2.normalize(image, None,beta=255, dtype=cv2.CV_32F)
image = cv2.resize(image, (224, 224))
image = np.expand_dims(image, axis=0)
This also didn't work, I also tried many other ways to predict a single image, but none worked.
Finally, the only way was that I had to create an Image Data Generator and Flow From Directory for this single image, and it worked, but I believe that's not how it should be done.
The code img = tf.keras.applications.vgg16.preprocess_input(img) scales the pixel
values in the image to values between -1 to +1 assuming the original pixel values are in the range 0 to 255. In the previous line of code
img = img.astype('float32') / 255.
You rescaled the pixels. So remove that line of code. Now to predict a single image you need to expand the dimensions with
img = np.expand_dims(img, axis=0)
In your second code effort be aware the CV2 reads in images as BGR. If your model was trained on RGB images then your predictions will be wrong. Use the code below to convert the image to RGB.
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
As a side note you can replace tf.keras.applications.vgg16.preprocess_input(img) with the function below which will scale the images between -1 to +1
def scalar(img):
return img/127.5 - 1
This answer could be one starting point:
Resnet50 produces different prediction when image loading and resizing is done with OpenCV
These are possible differences (short gist):
RGB vs BGR (OpenCV loads BGR)
The interpolation method used (INTER_LINEAR vs INTER_NEAREST).
img_to_array() transforms the data type into float32 rather than uint8 which is obtained by default when loading with OpenCV.
tf.keras.applications.vgg16.preprocess_input(img). This preprocessing function can actually differ from what you have written above as image preprocessing; it is also notable that, if you do not preprocess it while training in this particular way (preprocess_input()) then it also makes sense to have bad results on the test set, since the preprocessings are different.
Hope these observations shed some light.
I saw this code on the official keras documentation and I have read the images need to be resized/ scaled prior to feeding to model. Can you please advise?
from tensorflow.keras.applications.inception_v3 import InceptionV3
from tensorflow.keras.layers import Input
# input size
input_tensor = Input(shape=(150, 150, 3))
model = InceptionV3(input_tensor=input_tensor, weights='imagenet', include_top=True)
Inception V3 can work any size of image as long as your image has 3 channels. Because ImageNet images consist of 3 channels. The reason it can work with any size is that convolutions do not care about image-sizes. You can use it with also grayscale images with some extra work but I am not sure if it will destroy the network performance etc. For this, you need to set include_top = False, otherwise your image size should match with model's defined size, (299,299,3).
You can re-size images with Lambda layer. Let's say you have 1024x1024 images:
input_images = tf.keras.Input(shape=(1024, 1024, 3))
whatever_this_size = tf.keras.layers.Lambda(lambda x: tf.image.resize(x,(150,150),
method=tf.image.ResizeMethod.BILINEAR))(input_images)
model = InceptionV3(input_tensor=whatever_this_size, weights='imagenet', include_top=False)
If you are using TF-Dataset API, you can also do following:
your_train_data = your_train_data.map(lambda x, y: (tf.image.resize(x, (150,150), y))
I am working on image_classification problem(multi-class).
i am using resnet50 model( https://keras.io/applications/#classify-imagenet-classes-with-resnet50 ) along with pretrained db "imagenet" using keras
I am getting the the output labels for which the images i passed to the model.
But now,
i have image data and label data with me of my own dataset.
When i pass the images to the resnet50 model it gives back the imagenet labels that are already trained. Now, here, i want the output as my own labels which is already in dataset instead of getting imagenet labels.
How to to fine tune labels in resnet50 model with imagenet db in keras
I have tried the resnet50 model alone and it works fine. but, how to change the output to my own labels instead of imagenet pre-trained labels.
from keras.applications.resnet50 import ResNet50
from keras.preprocessing import image
from keras.applications.resnet50 import preprocess_input, decode_predictions
import numpy as np
import os
model = ResNet50(weights='imagenet')
path='/Users/resnet-sample/'
img_path=os.listdir(path)
count=0
for i in img_path:
img = image.load_img(path+i, target_size=(224, 224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
preds = model.predict(x)
print('Predicted:', decode_predictions(preds, top=1)[0], i)
count=count+1
print(preds)
example:
i have an elephant image in jpg format and label its as an 'elephant' in my dataset.
when i pass this image to resnet50 model which uses imagenet pre-trained db the output i received is 'African-Elephant'(imagenet-label).
So instead of getting imagenet label as output, i want to tune this as 'elephant' as label which is in my dataset.
So, not sure how to fine tune the last layers that gives output as my labels instead of imagenet labels.
Pelase help me on this.
Thanks,
Srknt73
The weights argument should be either None (random initialization), imagenet (pre-training on ImageNet), or the path to the weights file to be loaded. So you give the path to the file containing the labels of your dataset
After training model with ImageDataGenerator(1/255.), do I need to rescale image before predicting ?
I thought it is necessary but experiment result said NO.
I trained a Resnet50 model which has 37 class on top layer.
Model was trained with ImageDataGenerator like this.
datagen = ImageDataGenerator(rescale=1./255)
generator=datagen.flow_from_directory(
directory=os.path.join(os.getcwd(), data_folder),
target_size=(224,224),
batch_size=256,
classes=None,
class_mode='categorical')
history = model.fit_generator(generator, steps_per_epoch=generator.n / 256, epochs=10)
Accuracy achieved 98% after 10 epochs on my train dataset.
The problem is, when i tried to predict each image in TRAIN dataset, prediction was wrong ( result is 33 whatever input image was )
img_p = './data/pets/shiba_inu/shiba_inu_27.jpg'
img = cv2.imread(img_p, cv2.IMREAD_COLOR)
img = cv2.resize(img, (224,224))
img_arr = np.zeros((1,224,224,3))
img_arr[0, :, :, :] = img / 255.
pred = model.predict(img_arr)
yhat = np.argmax(pred, axis=1)
yhat is 5, but y is 33
When I replace this line
img_arr[0, :, :, :] = img / 255.
by this
img_arr[0, :, :, :] = img
yhat is exactly 33.
Someone might suggest to use predict_generator() instead of predict(), but I want to understand what I did wrong here.
I knew what's wrong here.
I'm using Imagenet pretrained model, which DO NOT rescale image by divide it to 255. I have to use resnet50.preprocess_input before train/test.
preprocess_input function can be found here.
https://github.com/keras-team/keras-applications/blob/master/keras_applications/imagenet_utils.py
You must do every preprocessing that you do on train data, on each data that you want to feed to your trained network. actually when, for example, you rescale train images and train a network, your network train to get a matrix with entries between 0 and 1 and find the proper category. so if after training phase, you feed an image without rescaling, you feed a matrix with entries between 0 and 255 to your trained network while your network did not learn how treat with such matrix.
If you are following pre-processing exactly same as at the time of training then, you might look at the part of your code where you are predicting class using yhat = np.argmax(pred, axis=1) my hunch is that there might be class mismatch in accordance to indexing, to check how your classes are indexed when you use flow_from_directory use class_map = generator.class_indices this will return you a dictionary which will show you how your classes are mapped against index.
Note: The reason I state this because I've faced similar problem, using Keras flow_from_directory doesn't sort classes and hence it's quite possible that your prediction class 1 lies on the index 10 while np.argmax will return you class 1'.
I'm trying to see the result of using ImageDataGenerator for data augmentation.
Keras reads the data but it seems that it doesn't perform any generation on them. I get as output:
Found 32 images belonging to 1 classes.
but no generated images are saved in the directory I mentioned in save_to_dir parameter of flow_from_directory method.
Here my code:
from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img
from keras import backend as K
K.set_image_dim_ordering('th')
#the path of images to apply augmentation on them
images_path='train'
#create an instance of ImageDataGenerator
datagen = ImageDataGenerator(width_shift_range=0.2,
height_shift_range=0.2)
datagen.flow_from_directory(directory=images_path, target_size=(480,752),color_mode='grayscale', class_mode=None, save_to_dir='saved',save_prefix='keras_')
img = load_img('train/images/photon10.png')
x = img_to_array(img)
x = x.reshape((1,) + x.shape)
datagen.flow(x,batch_size=1,save_to_dir='saved',save_format='png')
I even tried to perform augmentation on one image and it wasn't saved.
What could be the reason? I'm a starter with Keras.
Note: class mode is None because I don't have a specific category.
flow_from_directory() returns a DirectoryIterator. The files are not saved until you iterate over this iterator.
For example,
iterator = datagen.flow_from_directory(...)
next(iterator)
will save a batch of augmented images to save_to_dir. You can also use a for loop over the iterator to control how many images will be generated.
Its only a declaration, you must use that generator, for example, .next()
datagen.next()
then you will see images in saved