I'm trying to do an image segmentation problem where I want to segment 5 objects in an image. I'm using a U-net architecture. My final layer looks like this:
conv_final = Conv2D(OUTPUT_MASK_CHANNELS, (1, 1))(up_conv_224)
conv_final = Activation('sigmoid')(conv_final)
model = Model(inputs, conv_final, name="ZF_UNET_224")
However I get an error saying:
ValueError: Error when checking target: expected conv2d_24 to have shape (224, 224, 5) but got array with shape (224, 224, 3)
This is the generator that I'm using
image_generator = train_datagen.flow_from_directory(
'data/train', # this is the target directory
target_size=(224, 224),
color_mode = 'rgb',# all images will be resized to 150x150
batch_size=batch_size,
class_mode=None,
seed = 1) # since we use binary_crossentropy loss, we need binary labels
# this is a similar generator, for validation data
mask_generator = mask_datagen.flow_from_directory(
'data/train',
target_size=(224, 224),
color_mode = 'rgb',
batch_size=batch_size,
class_mode=None,
seed = 1)
train_generator = zip(image_generator, mask_generator)
What can I do to fix this? Any help appreciated!
You have to convert the data into one hot encoded format.
Use from keras.utils import to_categorical
Related
I'm trying to build a CNN model in order to classify an image, but whenever the training is done and I try to feed it a single image (from the training dataset) it misclassifies this image always.
Please take a look at the code I wrote below.
Thank you in advance.
First, I declared an Image Data Generator for both my training and testing sets:
train_datagen = ImageDataGenerator(rescale = 1./255, rotation_range=20, horizontal_flip = True,
validation_split=0.3)
test_datagen = ImageDataGenerator(rescale = 1./255,validation_split=0.3)
Then, I used the flow_from_directory() function to load the images:
train_generator = train_datagen.flow_from_directory(
data_dir,
shuffle=False,
subset='training',
target_size = (224, 224),
class_mode = 'categorical'
)
test_generator = test_datagen.flow_from_directory(
data_dir,
shuffle=False,
subset='validation',
target_size = (224, 224),
class_mode = 'categorical'
)
I then loaded a pretrained model and added a few layers to build my model:
pretrained_model = VGG16(weights="imagenet", include_top=False,
input_tensor=input_shape)
pretrained_model.trainable = False
model = tf.keras.Sequential([
pretrained_model,
Flatten(name="flatten"),
Dense(3, activation="softmax")
])
I then trained the model :
INIT_LR = 3e-4
EPOCHS = 15
opt = Adam(lr=INIT_LR)
model.compile(loss="categorical_crossentropy", optimizer='Adam', metrics=["accuracy"])
H = model.fit(
train_generator,
validation_data=test_generator,
epochs=EPOCHS,
verbose= 1)
Then came the part to predict a single image:
I chose an image that was part of the training set, I even overfitted the model to make sure the predictions should be correct, but it was giving me wrong results for every image I input to the model.
I tried the following ways:
image = image.load_img(url,target_size = (224, 224))
img = tf.keras.preprocessing.image.img_to_array(image)
img = np.array([img])
img = img.astype('float32') / 255.
img = tf.keras.applications.vgg16.preprocess_input(img)
This didn't work
image = cv2.imread(url)
image = cv2.normalize(image, None,beta=255, dtype=cv2.CV_32F)
image = cv2.resize(image, (224, 224))
image = np.expand_dims(image, axis=0)
This also didn't work, I also tried many other ways to predict a single image, but none worked.
Finally, the only way was that I had to create an Image Data Generator and Flow From Directory for this single image, and it worked, but I believe that's not how it should be done.
The code img = tf.keras.applications.vgg16.preprocess_input(img) scales the pixel
values in the image to values between -1 to +1 assuming the original pixel values are in the range 0 to 255. In the previous line of code
img = img.astype('float32') / 255.
You rescaled the pixels. So remove that line of code. Now to predict a single image you need to expand the dimensions with
img = np.expand_dims(img, axis=0)
In your second code effort be aware the CV2 reads in images as BGR. If your model was trained on RGB images then your predictions will be wrong. Use the code below to convert the image to RGB.
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
As a side note you can replace tf.keras.applications.vgg16.preprocess_input(img) with the function below which will scale the images between -1 to +1
def scalar(img):
return img/127.5 - 1
This answer could be one starting point:
Resnet50 produces different prediction when image loading and resizing is done with OpenCV
These are possible differences (short gist):
RGB vs BGR (OpenCV loads BGR)
The interpolation method used (INTER_LINEAR vs INTER_NEAREST).
img_to_array() transforms the data type into float32 rather than uint8 which is obtained by default when loading with OpenCV.
tf.keras.applications.vgg16.preprocess_input(img). This preprocessing function can actually differ from what you have written above as image preprocessing; it is also notable that, if you do not preprocess it while training in this particular way (preprocess_input()) then it also makes sense to have bad results on the test set, since the preprocessings are different.
Hope these observations shed some light.
I have a Multi-Layer Perceptron network in Keras with two hidden Layers.
While trying to train the network I get the Error in the fit_generator :
Error:
ValueError: logits and labels must have the same shape ((None, 2) vs (None, 1))
My Code is:
import numpy as np
import keras
from keras import layers
from keras import Sequential
# Define Window size (color images)
img_window = (32,32,3)
# Flatten the Window shape
input_shape = np.prod(img_window)
print(input_shape)
# Define MLP with two hidden layers(neurons)
simpleMLP = Sequential(
[layers.Input(shape=img_window),
layers.Flatten(), # Flattens the input, conv2D to 1 vector , which does not affect the batch size.
layers.Dense(input_shape//2 ,activation="relu"),
layers.Dense(input_shape//2 ,activation="relu"),
layers.Dense(2,activation="sigmoid"),
]
)
# After model is "built" call its summary() menthod to display its contents
simpleMLP.summary()
# Initialization
# Size of the batches of data, adjust it depends on RAM
batch_size = 128
epochs = 5
# Compile MLP model with 3 arguments: loss function, optimizer, and metrics function to judge model performance
simpleMLP.compile(loss="binary_crossentropy",optimizer="adam",metrics=["binary_accuracy"]) #BCE
# Create ImagedataGenerator to splite training, validation dataset
from keras.preprocessing.image import ImageDataGenerator
train_dir = '/content/train'
train_datagen = ImageDataGenerator(
rescale=1./255, # rescaling factor
shear_range=0.1,
zoom_range=0.1,
horizontal_flip=True,
fill_mode='nearest')
valid_dir = '/content/valid'
valid_datagen =ImageDataGenerator(
rescale=1./255,
shear_range=0.1,
zoom_range=0.1,
horizontal_flip=True,
fill_mode='nearest')
train_generator = train_datagen.flow_from_directory(
train_dir,
target_size=img_window[:2],
batch_size=batch_size,
class_mode='binary',
color_mode='rgb'
)
validation_generator = valid_datagen.flow_from_directory(
valid_dir,
target_size=img_window[:2],
batch_size=batch_size,
class_mode='binary',
color_mode='rgb')
# Train the MLP model
simpleMLP.fit_generator((
train_generator,
steps_per_epoch= 8271 // batch_size,
epochs=5,
validation_data=validation_generator,
validation_steps= 2072 // batch_size)
Can you please advise me how to resolve this problem? thanks in advance.
You problem simply is that, you have got labels of shape (N, 1) and loss defined as binary_crossentropy. This means you should have a single output node in the last layer. But you have a model that outputs two classes.
simpleMLP = Sequential(
[...
layers.Dense(2,activation="sigmoid"),
]
)
Simply change this to,
simpleMLP = Sequential(
[...
layers.Dense(1,activation="sigmoid"),
]
)
I am working on a project to classify waste as plastics and non plastics using only.images to train them.However i still dont know what features does the model take into account while classifyimg them.I am using CNN,however the accuracy of prediction is still not up to the mark.
The reason why i went to CNN because there is no specific feature to distinguish plastics from others.Is there any other way to approach this problem?
For eg If i train the images of cats,my Neural Network learns what is a cat however i do not explicitly give features is the same case valid here too?
Suppose you want to extract the Features from the Pre-Trained Convolutional Neural Network, VGGNet, VGG16.
Code to reuse the Convolutional Base is:
from keras.applications import VGG16
conv_base = VGG16(weights='imagenet',
include_top=False,
input_shape=(150, 150, 3)) # This is the Size of your Image
The final feature map has shape (4, 4, 512). That’s the feature on top of which you’ll stick a densely connected classifier.
There are 2 ways to extract Features:
FAST FEATURE EXTRACTION WITHOUT DATA AUGMENTATION: Running the convolutional base over your dataset, recording its output to a
Numpy array on disk, and then using this data as input to a standalone, densely
connected classifier similar to those you saw in part 1 of this book. This solution is fast and cheap to run, because it only requires running the convolutional base once for every input image, and the convolutional base is by far the most expensive part of the pipeline. But for the same reason, this technique won’t allow you to use data augmentation.
Code for extracting Features using this method is shown below:
import os
import numpy as np
from keras.preprocessing.image import ImageDataGenerator
base_dir = '/Users/fchollet/Downloads/cats_and_dogs_small'
train_dir = os.path.join(base_dir, 'train')
validation_dir = os.path.join(base_dir, 'validation')
test_dir = os.path.join(base_dir, 'test')
datagen = ImageDataGenerator(rescale=1./255)
batch_size = 20
def extract_features(directory, sample_count):
features = np.zeros(shape=(sample_count, 4, 4, 512))
labels = np.zeros(shape=(sample_count))
generator = datagen.flow_from_directory(directory, target_size=(150, 150),
batch_size=batch_size, class_mode='binary')
i=0
for inputs_batch, labels_batch in generator:
features_batch = conv_base.predict(inputs_batch)
features[i * batch_size : (i + 1) * batch_size] = features_batch
labels[i * batch_size : (i + 1) * batch_size] = labels_batch
i += 1
if i * batch_size >= sample_count:
break
return features, labels
train_features, train_labels = extract_features(train_dir, 2000)
validation_features, validation_labels = extract_features(validation_dir,1000)
test_features, test_labels = extract_features(test_dir, 1000)
train_features = np.reshape(train_features, (2000, 4*4* 512))
validation_features = np.reshape(validation_features, (1000, 4*4* 512))
test_features = np.reshape(test_features, (1000, 4*4* 512))
from keras import models
from keras import layers
from keras import optimizers
model = models.Sequential()
model.add(layers.Dense(256, activation='relu', input_dim=4 * 4 * 512))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(1, activation='sigmoid'))
model.compile(optimizer=optimizers.RMSprop(lr=2e-5),
loss='binary_crossentropy', metrics=['acc'])
history = model.fit(train_features, train_labels, epochs=30,
batch_size=20, validation_data=(validation_features, validation_labels))
Training is very fast, because you only have to deal with two Dense
layers—an epoch takes less than one second even on CPU
FEATURE EXTRACTION WITH DATA AUGMENTATION: Extending the model you have (conv_base) by adding Dense layers on top, and running the whole thing end to end on the input data. This will allow you to use data augmentation, because every input image goes through the convolutional base every time it’s seen by the model. But for the same reason, this technique is far more expensive than the first
Code for the same is shown below:
from keras import models
from keras import layers
model = models.Sequential()
model.add(conv_base)
model.add(layers.Flatten())
model.add(layers.Dense(256, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
from keras.preprocessing.image import ImageDataGenerator
from keras import optimizers
train_datagen = ImageDataGenerator(rescale=1./255,rotation_range=40,
width_shift_range=0.2,height_shift_range=0.2,shear_range=0.2,
zoom_range=0.2,horizontal_flip=True,fill_mode='nearest')
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(train_dir,target_size=(150, 150), batch_size=20, class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
validation_dir,
target_size=(150, 150),
batch_size=20,
class_mode='binary')
model.compile(loss='binary_crossentropy',
optimizer=optimizers.RMSprop(lr=2e-5),
metrics=['acc'])
history = model.fit_generator(train_generator, steps_per_epoch=100, epochs=30, validation_data=validation_generator, validation_steps=50)
For more details, please refer Section 5.3.1 of the book, "Deep Learning with Python", authored by Father of Keras, "Francois Chollet"
I have trained a basic CNN model for image classification.
While training the model I have used ImageDataGenerator from keras api.
After the model is being trained i used testdatagenerator and flow_from_directory method for testing.
Everything Went well.
Then I saved the model for future use.
Now i am using the same model and used predict method from keras api with a single image, but the prediction is very different every time I test using different images.
Could you please let me know any solution.
training_augmentation = ImageDataGenerator(rescale=1 / 255.0)
validation_testing_augmentation = ImageDataGenerator(rescale=1 / 255.0)
# Initialize training generator
training_generator = training_augmentation.flow_from_directory(
JPG_TRAIN_IMAGE_DIR,
class_mode="categorical",
target_size=(32, 32),
color_mode="rgb",
shuffle=True,
batch_size=batch_size
)
# initialize the validation generator
validation_generator = validation_testing_augmentation.flow_from_directory(
JPG_VAL_IMAGE_DIR,
class_mode="categorical",
target_size=(32, 32),
color_mode="rgb",
shuffle=False,
batch_size=batch_size
)
# initialize the testing generator
testing_generator = validation_testing_augmentation.flow_from_directory(
JPG_TEST_IMAGE_DIR,
class_mode="categorical",
target_size=(32, 32),
color_mode="rgb",
shuffle=False,
batch_size=batch_size
)
history = model.fit_generator(
training_generator,
steps_per_epoch=total_img_count_dict['train'] // batch_size,
validation_data=validation_generator,
validation_steps=total_img_count_dict['val'] // batch_size,
epochs=epochs,
callbacks=callbacks)
testing_generator.reset()
prediction_stats = model.predict_generator(testing_generator, steps=(total_img_count_dict['test'] // batch_size) + 1)
### Trying to use predict method
img_file = '/content/drive/My Drive/Traffic_Sign_Recognition/to_display_output/Copy of 00003_00019.jpg'
img = cv2.imread(img_file)
img=cv2.resize(img, (32,32))
img = img/255.0
a=np.reshape(img, (1, 32, 32, 3))
model = load_model('/content/drive/My Drive/Traffic_Sign_Recognition/basic_cnn.h5')
prediction = model.predict(a)
When I am trying to use predict, every time wrong prediction is coming.
Any leads will be appreciated.
Keras generator uses PIL for image reading which read images from disk as RGB.
You are using opencv for reading which reads images as BGR. You have to convert your image from BGR to RGB.
img = cv2.imread(img_file)
img = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
...
I am using a pre-trained Resnet50 model for simple feature extraction for images. but it gives me this error.
Error when checking input: expected input_9 to have the shape (224, 224, 3) but got array with shape (244, 244, 3)
I thought I changed the shape correctly and added a dimension to it like this tutorial said to do. https://www.kaggle.com/kelexu/extract-resnet-feature-using-keras
But it still gives me the above error.
What am I doing wrong here?
# load pre-trained resnet50
base_model = ResNet50(weights='imagenet', include_top=False,pooling=max)
x = base_model.output
input = Input(shape=(224,224,3))
x = Flatten()(input)
model = Model(inputs=input, outputs=x)
# Load in image
img = image.load_img("001.png", target_size=(244, 244))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
print(x.shape) # This produces (1, 244, 244, 3)
features = model.predict(x)
features_reduce = features.squeeze()
Change
img = image.load_img("001.png", target_size=(244, 244))
to
img = image.load_img("001.png", target_size=(224, 224))