How to extract features from an image for training a CNN model - python-3.x

I am working on a project to classify waste as plastics and non plastics using only.images to train them.However i still dont know what features does the model take into account while classifyimg them.I am using CNN,however the accuracy of prediction is still not up to the mark.
The reason why i went to CNN because there is no specific feature to distinguish plastics from others.Is there any other way to approach this problem?
For eg If i train the images of cats,my Neural Network learns what is a cat however i do not explicitly give features is the same case valid here too?

Suppose you want to extract the Features from the Pre-Trained Convolutional Neural Network, VGGNet, VGG16.
Code to reuse the Convolutional Base is:
from keras.applications import VGG16
conv_base = VGG16(weights='imagenet',
include_top=False,
input_shape=(150, 150, 3)) # This is the Size of your Image
The final feature map has shape (4, 4, 512). That’s the feature on top of which you’ll stick a densely connected classifier.
There are 2 ways to extract Features:
FAST FEATURE EXTRACTION WITHOUT DATA AUGMENTATION: Running the convolutional base over your dataset, recording its output to a
Numpy array on disk, and then using this data as input to a standalone, densely
connected classifier similar to those you saw in part 1 of this book. This solution is fast and cheap to run, because it only requires running the convolutional base once for every input image, and the convolutional base is by far the most expensive part of the pipeline. But for the same reason, this technique won’t allow you to use data augmentation.
Code for extracting Features using this method is shown below:
import os
import numpy as np
from keras.preprocessing.image import ImageDataGenerator
base_dir = '/Users/fchollet/Downloads/cats_and_dogs_small'
train_dir = os.path.join(base_dir, 'train')
validation_dir = os.path.join(base_dir, 'validation')
test_dir = os.path.join(base_dir, 'test')
datagen = ImageDataGenerator(rescale=1./255)
batch_size = 20
def extract_features(directory, sample_count):
features = np.zeros(shape=(sample_count, 4, 4, 512))
labels = np.zeros(shape=(sample_count))
generator = datagen.flow_from_directory(directory, target_size=(150, 150),
batch_size=batch_size, class_mode='binary')
i=0
for inputs_batch, labels_batch in generator:
features_batch = conv_base.predict(inputs_batch)
features[i * batch_size : (i + 1) * batch_size] = features_batch
labels[i * batch_size : (i + 1) * batch_size] = labels_batch
i += 1
if i * batch_size >= sample_count:
break
return features, labels
train_features, train_labels = extract_features(train_dir, 2000)
validation_features, validation_labels = extract_features(validation_dir,1000)
test_features, test_labels = extract_features(test_dir, 1000)
train_features = np.reshape(train_features, (2000, 4*4* 512))
validation_features = np.reshape(validation_features, (1000, 4*4* 512))
test_features = np.reshape(test_features, (1000, 4*4* 512))
from keras import models
from keras import layers
from keras import optimizers
model = models.Sequential()
model.add(layers.Dense(256, activation='relu', input_dim=4 * 4 * 512))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(1, activation='sigmoid'))
model.compile(optimizer=optimizers.RMSprop(lr=2e-5),
loss='binary_crossentropy', metrics=['acc'])
history = model.fit(train_features, train_labels, epochs=30,
batch_size=20, validation_data=(validation_features, validation_labels))
Training is very fast, because you only have to deal with two Dense
layers—an epoch takes less than one second even on CPU
FEATURE EXTRACTION WITH DATA AUGMENTATION: Extending the model you have (conv_base) by adding Dense layers on top, and running the whole thing end to end on the input data. This will allow you to use data augmentation, because every input image goes through the convolutional base every time it’s seen by the model. But for the same reason, this technique is far more expensive than the first
Code for the same is shown below:
from keras import models
from keras import layers
model = models.Sequential()
model.add(conv_base)
model.add(layers.Flatten())
model.add(layers.Dense(256, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
from keras.preprocessing.image import ImageDataGenerator
from keras import optimizers
train_datagen = ImageDataGenerator(rescale=1./255,rotation_range=40,
width_shift_range=0.2,height_shift_range=0.2,shear_range=0.2,
zoom_range=0.2,horizontal_flip=True,fill_mode='nearest')
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(train_dir,target_size=(150, 150), batch_size=20, class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
validation_dir,
target_size=(150, 150),
batch_size=20,
class_mode='binary')
model.compile(loss='binary_crossentropy',
optimizer=optimizers.RMSprop(lr=2e-5),
metrics=['acc'])
history = model.fit_generator(train_generator, steps_per_epoch=100, epochs=30, validation_data=validation_generator, validation_steps=50)
For more details, please refer Section 5.3.1 of the book, "Deep Learning with Python", authored by Father of Keras, "Francois Chollet"

Related

ValueError: logits and labels must have the same shape

I have a Multi-Layer Perceptron network in Keras with two hidden Layers.
While trying to train the network I get the Error in the fit_generator :
Error:
ValueError: logits and labels must have the same shape ((None, 2) vs (None, 1))
My Code is:
import numpy as np
import keras
from keras import layers
from keras import Sequential
# Define Window size (color images)
img_window = (32,32,3)
# Flatten the Window shape
input_shape = np.prod(img_window)
print(input_shape)
# Define MLP with two hidden layers(neurons)
simpleMLP = Sequential(
[layers.Input(shape=img_window),
layers.Flatten(), # Flattens the input, conv2D to 1 vector , which does not affect the batch size.
layers.Dense(input_shape//2 ,activation="relu"),
layers.Dense(input_shape//2 ,activation="relu"),
layers.Dense(2,activation="sigmoid"),
]
)
# After model is "built" call its summary() menthod to display its contents
simpleMLP.summary()
# Initialization
# Size of the batches of data, adjust it depends on RAM
batch_size = 128
epochs = 5
# Compile MLP model with 3 arguments: loss function, optimizer, and metrics function to judge model performance
simpleMLP.compile(loss="binary_crossentropy",optimizer="adam",metrics=["binary_accuracy"]) #BCE
# Create ImagedataGenerator to splite training, validation dataset
from keras.preprocessing.image import ImageDataGenerator
train_dir = '/content/train'
train_datagen = ImageDataGenerator(
rescale=1./255, # rescaling factor
shear_range=0.1,
zoom_range=0.1,
horizontal_flip=True,
fill_mode='nearest')
valid_dir = '/content/valid'
valid_datagen =ImageDataGenerator(
rescale=1./255,
shear_range=0.1,
zoom_range=0.1,
horizontal_flip=True,
fill_mode='nearest')
train_generator = train_datagen.flow_from_directory(
train_dir,
target_size=img_window[:2],
batch_size=batch_size,
class_mode='binary',
color_mode='rgb'
)
validation_generator = valid_datagen.flow_from_directory(
valid_dir,
target_size=img_window[:2],
batch_size=batch_size,
class_mode='binary',
color_mode='rgb')
# Train the MLP model
simpleMLP.fit_generator((
train_generator,
steps_per_epoch= 8271 // batch_size,
epochs=5,
validation_data=validation_generator,
validation_steps= 2072 // batch_size)
Can you please advise me how to resolve this problem? thanks in advance.
You problem simply is that, you have got labels of shape (N, 1) and loss defined as binary_crossentropy. This means you should have a single output node in the last layer. But you have a model that outputs two classes.
simpleMLP = Sequential(
[...
layers.Dense(2,activation="sigmoid"),
]
)
Simply change this to,
simpleMLP = Sequential(
[...
layers.Dense(1,activation="sigmoid"),
]
)

Keras ImageDataGenerator Low Validation Accuracy

I'm trying to preform transfer learning with mobilenetv2 to classify 196 classes of cars from the cars-196 dataset of stanford.
My work environment is google colab notebook.
I use the ImageDataGenerator from keras to load the images for the train and validation.
On the training images I also perform data augmentation.
The following code is how I perform it:
# To load the dataset from the drive
from google.colab import drive
drive.mount('/content/drive')
import math
from keras.models import Sequential, Model
from keras.layers import Dense, Conv2D, MaxPooling2D, Flatten, Dropout, ReLU, GlobalAveragePooling2D
from keras.preprocessing.image import ImageDataGenerator
from keras.applications.mobilenet_v2 import MobileNetV2, preprocess_input
BATCH_SIZE = 196
train_datagen = ImageDataGenerator(
rotation_range=20, # Rotate the augmented image by 20 degrees
zoom_range=0.2, # Zoom by 20% more or less
horizontal_flip=True, # Allow for horizontal flips of augmented images
brightness_range=[0.8, 1.2], # Lighter and darker images by 20%
width_shift_range=0.1,
height_shift_range=0.1,
preprocessing_function=preprocess_input
)
img_data_iterator = train_datagen.flow_from_directory(
# Where to take the data from, the classes are the sub folder names
'/content/drive/My Drive/Datasets/cars-196/car_data/train',
class_mode="categorical", # classes are in 2D one hot encoded way, default is true but just to point it out
shuffle=True, # shuffle the data, default is true but just to point it out
batch_size=BATCH_SIZE,
target_size=(224, 224) # This size is the default of mobilenet NN
)
validation_img_data_iterator = ImageDataGenerator().flow_from_directory(
'/content/drive/My Drive/Datasets/cars-196/car_data/test',
class_mode="categorical",
shuffle=True,
batch_size=BATCH_SIZE,
target_size=(224, 224)
)
base_model = MobileNetV2(weights='imagenet', include_top=False)
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(512, activation='relu')(x)
x = Dropout(0.5)(x)
preds = Dense(196, activation='softmax')(x)
model = Model(inputs=base_model.input, outputs=preds)
# Disable training of already trained layer
for layer in model.layers[:-3]:
layer.trainable = False
model.compile(optimizer='Adam',loss='categorical_crossentropy',metrics=['accuracy'])
# define the checkpoint
from keras.callbacks import ModelCheckpoint
filepath = "/content/drive/My Drive/Datasets/cars-196/model.h5"
checkpoint = ModelCheckpoint(filepath, monitor='loss', verbose=1, save_best_only=True, mode='min')
callbacks_list = [checkpoint]
history = model.fit(
img_data_iterator,
steps_per_epoch=math.ceil(8144/BATCH_SIZE), # 8144 is the number of training images
validation_steps=math.ceil(8062/BATCH_SIZE), # 8062 is the number of validation images
validation_data=validation_img_data_iterator,
epochs=100,
callbacks=callbacks_list
)
About the batch size, from this stackoverflow question I decided to set the batch size as the number of labels available, but it did not change anything in terms of val_accuracy.
I've added a dropout of 0.5 between the fully connected layers that I have added, but again, no change in the accuracy of the validation.
My accuracy on the training set arrives to about 92% while the validation accuracy stays at about 0.7%.
My guess is that the ImageDataGenerator is acting weird and screwing up the accuracy, but I have not found any solution for the problem so ATM I dont have a clew what is the reason behind it.
Does anyone have any guesses as to what might be the problem?
----- EDIT
The train and test folder all have sub folders with name of the labels (the different cars I want to identify) and each subfolder has images of that car. This is just how the cars-196 dataset is. The ImageDataGenerator attaches the right label to the image, depending on in what subfolder that image was in.
The problem was that I did not apply the preprocess_input function to the validation data image generator.
Instead of
validation_img_data_iterator = ImageDataGenerator().flow_from_directory(
'/content/drive/My Drive/Datasets/cars-196/car_data/test',
class_mode="categorical",
shuffle=True,
batch_size=BATCH_SIZE,
target_size=(224, 224)
)
Changed it to
validation_img_data_iterator = ImageDataGenerator(
preprocessing_function=preprocess_input
).flow_from_directory(
'/content/drive/My Drive/Datasets/cars-196/car_data/test',
class_mode="categorical",
shuffle=True,
batch_size=BATCH_SIZE,
target_size=(224, 224)
)

Unable to load the saved model in the browser using Tensorflowjs

I am trying to build a rice classifier using transfer learning on edge device, I took the help of tutorial at https://github.com/ADLsourceCode/TensorflowJS
My sample data is at https://www.dropbox.com/s/esirpr6q1lsdsms/ricetransfer1.zip?dl=0
I saved the model locally using the code mentioned below for rice classification and kept in folder TensorflowJS/Mobilenet_VGG16_Keras_To_TensorflowJS/static/
alongwith vgg and mobilenet but the, I am not able to load the rice model on tensorflowjs in the browser.
If I trying the save the vgg model in my local system and load the model in the tensoflowjs(in browser) it's working well.
# Base variables
import os
base_dir = 'ricetransfer1/'
train_dir = os.path.join(base_dir, 'train')
validation_dir = os.path.join(base_dir, 'validation')
test_dir = os.path.join(base_dir, 'test')
train_cats_dir = os.path.join(train_dir, 'KN')
train_dogs_dir = os.path.join(train_dir, 'DM')
train_size, validation_size, test_size = 90, 28, 26
#train_size, validation_size, test_size = 20, 23, 14
img_width, img_height = 224, 224 # Default input size for VGG16
# Instantiate convolutional base
from keras.applications import VGG16
import tensorflowjs as tfjs
import tensorflow as tf
tf.compat.v1.disable_eager_execution()
img_width, img_height = 224, 224 # Default input size for VGG16
conv_base = VGG16(weights='imagenet',
include_top=False,
input_shape=(img_width, img_height, 3))
# 3 = number of channels in RGB pictures
#saving the vgg model to run it locally
tfjs.converters.save_keras_model(conv_base, '/TensorflowJS/Mobilenet_VGG16_Keras_To_TensorflowJS/static/vgg')
# Check architecture
conv_base.summary()
# Extract features
import os, shutil
from keras.preprocessing.image import ImageDataGenerator
import numpy as np
train_size, validation_size, test_size = 90, 28, 25
datagen = ImageDataGenerator(rescale=1./255)
batch_size = 1
#train_dir = "ricetransfer1/train"
#validation_dir = "ricetransfer1/validation"
#test_dir="ricetransfer1/test"
#indices = np.random.choice(range(len(X_train)))
def extract_features(directory, sample_count):
#sample_count= X_train.ravel()
features = np.zeros(shape=(sample_count, 7, 7, 512)) # Must be equal to the output of the convolutional base
labels = np.zeros(shape=(sample_count))
# Preprocess data
generator = datagen.flow_from_directory(directory,
target_size=(img_width,img_height),
batch_size = batch_size,
class_mode='binary')
# Pass data through convolutional base
i = 0
for inputs_batch, labels_batch in generator:
features_batch = conv_base.predict(inputs_batch)
features[i * batch_size: (i + 1) * batch_size] = features_batch
labels[i * batch_size: (i + 1) * batch_size] = labels_batch
i += 1
if i * batch_size >= sample_count:
break
return features, labels
train_features, train_labels = extract_features(train_dir, train_size) # Agree with our small dataset size
validation_features, validation_labels = extract_features(validation_dir, validation_size)
test_features, test_labels = extract_features(test_dir, test_size)
# Define model
from keras import models
from keras import layers
from keras import optimizers
epochs = 2
ricemodel = models.Sequential()
ricemodel.add(layers.Flatten(input_shape=(7,7,512)))
ricemodel.add(layers.Dense(256, activation='relu', input_dim=(7*7*512)))
ricemodel.add(layers.Dropout(0.5))
ricemodel.add(layers.Dense(1, activation='sigmoid'))
ricemodel.summary()
# Compile model
ricemodel.compile(optimizer=optimizers.Adam(),
loss='binary_crossentropy',
metrics=['acc'])
# Train model
import os
history = ricemodel.fit(train_features, train_labels,
epochs=epochs,
batch_size=batch_size,
validation_data=(validation_features, validation_labels))
##saving the rice classification model to run it locally
tfjs.converters.save_keras_model(ricemodel, '/TensorflowJS/Mobilenet_VGG16_Keras_To_TensorflowJS/static/rice/')
I think, there is some mistake in the rice model, how can I solve the issue?
The expected output is to run the rice classification on the browser using tensorflowjs
I think here it might be you are getting an error due to the older version of the tfjs file.
update the latest version to
<script src="https://cdn.jsdelivr.net/npm/#tensorflow/tfjs#0.13.5"></script>
in your html page but it might arise a new error to due different image size.
I will suggest the open the develope mode in the browser to see the exact error, in this it worked.

MNIST and transfer learning with VGG16 in Keras- low validation accuracy

I recently started taking advantage of Keras's flow_from_dataframe() feature for a project, and decided to test it with the MNIST dataset. I have a directory full of the MNIST samples in png format, and a dataframe with the absolute directory for each in one column and the label in the other.
I'm also using transfer learning, importing VGG16 as a base, and adding my own 512 node relu dense layer and 0.5 drop-out before a softmax layer of 10. (For digits 0-9). I'm using rmsprop (lr=1e-4) as the optimizer.
When I launch my environment, it calls the latest version of keras_preprocessing from Git, which has support for absolute directories and capitalized file extensions.
My problem is that I have a very high training accuracy, and a terribly low validation accuracy. By my final epoch (10), I had a training accuracy of 0.94 and a validation accuracy of 0.01.
I'm wondering if there's something fundamentally wrong with my script? With another dataset, I'm even getting NaNs for both my training and validation loss values after epoch 4. (I checked the relevant columns, there aren't any null values!)
Here's my code. I'd be deeply appreciative is someone could glance through it and see if anything jumped out at them.
import pandas as pd
import numpy as np
import keras
from keras_preprocessing.image import ImageDataGenerator
from keras import applications
from keras import optimizers
from keras.models import Model
from keras.layers import Dropout, Flatten, Dense, GlobalAveragePooling2D
from keras import backend as k
from keras.callbacks import ModelCheckpoint, CSVLogger
from keras.applications.vgg16 import VGG16, preprocess_input
# INITIALIZE MODEL
img_width, img_height = 32, 32
model = VGG16(weights = 'imagenet', include_top=False, input_shape = (img_width, img_height, 3))
# freeze all layers
for layer in model.layers:
layer.trainable = False
# Adding custom Layers
x = model.output
x = Flatten()(x)
x = Dense(512, activation='relu')(x)
x = Dropout(0.5)(x)
predictions = Dense(10, activation="softmax")(x)
# creating the final model
model_final = Model(input = model.input, output = predictions)
# compile the model
rms = optimizers.RMSprop(lr=1e-4)
#adadelta = optimizers.Adadelta(lr=0.001, rho=0.5, epsilon=None, decay=0.0)
model_final.compile(loss = "categorical_crossentropy", optimizer = rms, metrics=["accuracy"])
# LOAD AND DEFINE SOURCE DATA
train = pd.read_csv('MNIST_train.csv', index_col=0)
val = pd.read_csv('MNIST_test.csv', index_col=0)
nb_train_samples = 60000
nb_validation_samples = 10000
batch_size = 60
epochs = 10
# Initiate the train and test generators
train_datagen = ImageDataGenerator()
test_datagen = ImageDataGenerator()
train_generator = train_datagen.flow_from_dataframe(dataframe=train,
directory=None,
x_col='train_samples',
y_col='train_labels',
has_ext=True,
target_size = (img_height,
img_width),
batch_size = batch_size,
class_mode = 'categorical',
color_mode = 'rgb')
validation_generator = test_datagen.flow_from_dataframe(dataframe=val,
directory=None,
x_col='test_samples',
y_col='test_labels',
has_ext=True,
target_size = (img_height,
img_width),
batch_size = batch_size,
class_mode = 'categorical',
color_mode = 'rgb')
# GET CLASS INDICES
print('****************')
for cls, idx in train_generator.class_indices.items():
print('Class #{} = {}'.format(idx, cls))
print('****************')
# DEFINE CALLBACKS
path = './chk/epoch_{epoch:02d}-valLoss_{val_loss:.2f}-valAcc_{val_acc:.2f}.hdf5'
chk = ModelCheckpoint(path, monitor = 'val_acc', verbose = 1, save_best_only = True, mode = 'max')
logger = CSVLogger('./chk/training_log.csv', separator = ',', append=False)
nPlus = 1
samples_per_epoch = nb_train_samples * nPlus
# Train the model
model_final.fit_generator(train_generator,
steps_per_epoch = int(samples_per_epoch/batch_size),
epochs = epochs,
validation_data = validation_generator,
validation_steps = int(nb_validation_samples/batch_size),
callbacks = [chk, logger])
Have you tried explicitly defining the classes of the images? as such:
train_generator=image.ImageDataGenerator().flow_from_dataframe(classes=[0,1,2,3,4,5,6,7,8,9])
in both the train and validation generators.
I have found that sometimes the train and validation generators create different correspondence dictionaries.

Data Augmentation hurts accuracy Keras

I'm trying to adapt Deep Learning with Python section 5.3 Feature extraction with Data Augmentation to a 3-class problem with resnet50 (imagenet weights).
Full code at https://github.com/morenoh149/plantdisease
from keras import models
from keras import layers
from keras.applications.resnet50 import ResNet50
from keras import optimizers
from keras.preprocessing.image import ImageDataGenerator
input_shape = (224, 224, 3)
target_size = (224, 224)
batch_size = 20
conv_base = ResNet50(weights='imagenet', input_shape=input_shape, include_top=False)
model = models.Sequential()
model.add(conv_base)
model.add(layers.Flatten())
model.add(layers.Dense(256, activation='relu'))
model.add(layers.Dense(3, activation='softmax'))
conv_base.trainable = False
train_datagen = ImageDataGenerator(rescale=1./255)
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
'input/train',
target_size=target_size,
batch_size=batch_size,
class_mode='categorical')
validation_generator = test_datagen.flow_from_directory(
'input/validation',
target_size=target_size,
batch_size=batch_size,
class_mode='categorical')
model.compile(loss='categorical_crossentropy',
optimizer=optimizers.RMSprop(lr=2e-5),
metrics=['acc'])
history = model.fit_generator(
train_generator,
steps_per_epoch=96,
epochs=30,
verbose=2,
validation_data=validation_generator,
validation_steps=48)
Questions:
the book doesn't go much into ImageDataGenerator and selecting steps_per_epoch and validation_steps. What should these values be?
I have 3 classes, 1000 images each. I've split it 60/20/20 train/validation/test.
I was able to get a validation accuracy of 60% without data augmentation. Above I've simplified the ImageDataGenerator to only rescale. This model has a validation accuracy of 30% Why?
What changes do I need to make to the data-augmented version of this script to match the accuracy with no augmentation?
UPDATE:
This may be an issue with keras itself
https://github.com/keras-team/keras/issues/9214
https://github.com/keras-team/keras/pull/9965
To answer your first question: steps_per_epochis the number of batches the training generator should yield before considering an epoch finished. If you have 600 training images with batch size 20, this would be 30 steps per epoch et cetera. validation_steps applies the same logic to the validation data generator, be it at the end of each epoch.
In general, steps_per_epoch is the size of your dataset divided by the batch size.

Resources