Hello I am trying to build an image classifier using Keras and CNN
I already trained a model for Binary classification and it works really well.
I applied the same knowledge to build a Image classification using multiple categories(Which is failing miserably)
I have 5 classes I have created 5 folders inside jpeg dir and the directoy structure is as follows
C:\Users\jpeg
1.train
2.test
Inside train folder I have 5 subfolders each folder corresponding to a class
C:\Users\jpeg\train
1.Auth_Docs
2.Certificates_Reports
3.Document
4.Title
5.communication
and i placed appropriate images in each folder
Followed the exact same structure in test folder as well
source code:
import matplotlib.pyplot as plt
import cv2
%matplotlib inline
from keras.preprocessing.image import ImageDataGenerator
image_gen.flow_from_directory('C://Users/Jpeg/train')
image_gen.flow_from_directory('C://Users/jpeg/test')
image_shape = (150,150,3)
from keras.models import Sequential
from keras.layers import Activation, Dropout, Flatten, Dense, Conv2D, MaxPooling2D
model = Sequential()
model.add(Conv2D(filters=32, kernel_size=(3,3),input_shape=(150,150,3), activation='relu',))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(filters=64, kernel_size=(3,3),input_shape=(150,150,3), activation='relu',))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(filters=64, kernel_size=(3,3),input_shape=(150,150,3), activation='relu',))
model.add(MaxPooling2D(pool_size=(2, 2)))
#
model.add(Flatten())
#hidden layer number of neurons
model.add(Dense(256, activation='relu'))
# Here we say randomly turn off 30% of neurons.
model.add(Dropout(0.3))
# Last layer(add number of layers based on number of categories)
model.add(Dense(5, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
#Training the model
batch_size = 16
train_image_gen = image_gen.flow_from_directory('C://Users/jpeg/train',
target_size=image_shape[:2],
batch_size=batch_size,
class_mode='categorical'
)
#Found 2434 images belonging to 5 classes.
test_image_gen = image_gen.flow_from_directory('C://Users/jpeg/test',
target_size=image_shape[:2],
batch_size=batch_size,
class_mode='categorical'
)
#Found 60 images belonging to 5 classes.
train_image_gen.class_indices
#o/p
{'Auth_Docs': 0,
'Certificates_Reports': 1,
'Document': 2,
'Title': 3,
'communication': 4}
#Fitting the model
from PIL import Image
Image.MAX_IMAGE_PIXELS = None
results = model.fit_generator(train_image_gen,epochs=50,
steps_per_epoch=100,
validation_data=test_image_gen,
validation_steps=12)
#saving the model
model.save('Document_Classification.h5')
#results.accuracy for my model gives around 80% of accuracy
Now the issue with testing the model
from keras.models import load_model
new_model = load_model('Document_Classification.h5')
import numpy as np
from keras.preprocessing import image
import os,sys
from PIL import Image
Image.MAX_IMAGE_PIXELS = None
for a,b,c in os.walk("C:/Users/jpeg/test/communication"):
for i in c:
doc_img = image.load_img(os.path.join(a,i), target_size=(150, 150))
doc_img = image.img_to_array(doc_img)
doc_img = np.expand_dims(doc_img, axis=0)
doc_img = doc_img/255
#print (a,i)
prediction_prob = new_model.predict_classes(doc_img)
print(prediction_prob )
only output I get is
[2]
[2]
[2]
[2]
no matter which folder i use to test the o/p is the same a i.e in above example i used the communication folder images and the o/p is 2
same o/p when i test images from Auth_Docs,Title etc.
I don not see anything wrong in my code as this code worked for binary classification. Please advise
Also, I want to find what is the associated label with the output I am getting.
Please advise.
Thanks.
There are so many things you could do to troubleshoot. The amount of samples really matters; you should know this. Well, if I thought I had enough samples I'd save the images from the generators to check if they are ok (flow_from_directory - save_to_dir argument).
https://keras.io/preprocessing/image/
Also, while you are trainning, you could check tensorboard using callbacks (if you are using tensorflow) to see how bad/good your learning is going. Have a look at this video. See that what matters the most is val_acc.
Related
i am new to nlp and trying to learn the skip gram from the site:
https://towardsdatascience.com/understanding-feature-engineering-part-4-deep-learning-methods-for-text-data-96c44370bbfa
I am trying to implement the skip gram and the problem i run into is that the code below is a sequential API of keras and it doesn't support the merge ( later in the code as show below)
word_model.add(Embedding(vocab_size, embed_size,
embeddings_initializer="glorot_uniform",
input_length=1))
word_model.add(Reshape((embed_size, )))
so i am trying to convert it to functional api
word_model = Embedding(input_dim=vocab_size, output_dim=embed_size,
embeddings_initializer="glorot_uniform",
input_length=1)
word_model = Reshape(target_shape= (embed_size,))(word_model)
however i am getting the below error
Unexpectedly found an instance of type <class 'keras.layers.embeddings.Embedding'>. Expected a symbolic tensor instance.
i have tried reshape of layer and also background but still not working.
please suggest how to convert this or make it work.
thanks in advance.
from keras.layers import Merge
from keras.layers.core import Dense, Reshape
from keras.layers.embeddings import Embedding
from keras.models import Sequential
# build skip-gram architecture
word_model = Sequential()
word_model.add(Embedding(vocab_size, embed_size,
embeddings_initializer="glorot_uniform",
input_length=1))
word_model.add(Reshape((embed_size, )))
context_model = Sequential()
context_model.add(Embedding(vocab_size, embed_size,
embeddings_initializer="glorot_uniform",
input_length=1))
context_model.add(Reshape((embed_size,)))
model = Sequential()
model.add(Merge([word_model, context_model], mode="dot"))
model.add(Dense(1, kernel_initializer="glorot_uniform", activation="sigmoid"))
model.compile(loss="mean_squared_error", optimizer="rmsprop")
# view model summary
print(model.summary())
# visualize model structure
from IPython.display import SVG
from keras.utils.vis_utils import model_to_dot
SVG(model_to_dot(model, show_shapes=True, show_layer_names=False,
rankdir='TB').create(prog='dot', format='svg'))
You need an input layer first and then pass that on to the embedding layer. The following is an example using two inputs (one for the target word and one for the context word):
target_input = keras.layers.Input(input_shape)
context_input = keras.layers.Input(input_shape)
target_emb = Embedding(input_dim=vocab_size, output_dim=embed_size,
embeddings_initializer="glorot_uniform",
input_length=1)(target_input)
target_emb = Reshape((embed_size,))(target_emb)
context_emb = Embedding(input_dim=vocab_size, output_dim=embed_size,
embeddings_initializer="glorot_uniform",
input_length=1)(context_input)
context_emb = Reshape((embed_size,))(target_emb)
# Add the remaining layers here...
model = keras.models.Model(inputs=[target_input, context_input], outputs=output)
I am giving variable size images (all 278 images of different size 139 of each category) input to my cnn model. As a fact that cnn required fixed size images, so from here i got solution for this is to make input_shape=(None,Nonen,1) (for tensorflow backend and gray scale). but this solution doesnot work with flatten layer, so from their only i got solution of using GlobleMaxpooling or Globalaveragepooling. So from uses these facrts i am making a cnn model in keras to train my network with following code:
import os,cv2
import numpy as np
from sklearn.utils import shuffle
from keras import backend as K
from keras.utils import np_utils
from keras.models import Sequential
from keras.optimizers import SGD,RMSprop,adam
from keras.layers import Conv2D, MaxPooling2D,BatchNormalization,GlobalAveragePooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import regularizers
from keras import initializers
from skimage.io import imread_collection
from keras.preprocessing import image
from keras import Input
import keras
from keras import backend as K
#%%
PATH = os.getcwd()
# Define data path
data_path = PATH+'/current_exp'
data_dir_list = os.listdir(data_path)
img_rows=None
img_cols=None
num_channel=1
# Define the number of classes
num_classes = 2
img_data_list=[]
for dataset in data_dir_list:
img_list=os.listdir(data_path+'/'+ dataset)
print ('Loaded the images of dataset-'+'{}\n'.format(dataset))
for img in img_list:
input_img=cv2.imread(data_path + '/'+ dataset + '/'+ img,0)
img_data_list.append(input_img)
img_data = np.array(img_data_list)
if num_channel==1:
if K.image_dim_ordering()=='th':
img_data= np.expand_dims(img_data, axis=1)
print (img_data.shape)
else:
img_data= np.expand_dims(img_data, axis=4)
print (img_data.shape)
else:
if K.image_dim_ordering()=='th':
img_data=np.rollaxis(img_data,3,1)
print (img_data.shape)
#%%
num_classes = 2
#Total 278 sample, 139 for 0 category and 139 for category 1
num_of_samples = img_data.shape[0]
labels = np.ones((num_of_samples,),dtype='int64')
labels[0:138]=0
labels[138:]=1
x,y = shuffle(img_data,labels, random_state=2)
y = keras.utils.to_categorical(y, 2)
model = Sequential()
model.add(Conv2D(32,(2,2),input_shape=(None,None,1),activation='tanh',kernel_initializer=initializers.glorot_uniform(seed=100)))
model.add(Conv2D(32, (2,2),activation='tanh'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (2,2),activation='tanh'))
model.add(Conv2D(64, (2,2),activation='tanh'))
model.add(MaxPooling2D())
model.add(Dropout(0.25))
#model.add(Flatten())
model.add(GlobalAveragePooling2D())
model.add(Dense(256,activation='tanh'))
model.add(Dropout(0.25))
model.add(Dense(2,activation='softmax'))
model.compile(loss='categorical_crossentropy',optimizer='rmsprop',metrics=['accuracy'])
model.fit(x, y,batch_size=1,epochs=5,verbose=1)
but i am getting following error:
ValueError: Error when checking input: expected conv2d_1_input to have 4 dimensions, but got array with shape (278, 1)
how to solve it.
In the docs for Conv2D it says that the input tensor has to be in this format:
(samples, channels, rows, cols)
I believe you can't have a variable input size unless your network is fully convolutional.
Maybe what you want to do is to keep it to a fixed input size, and just resize the image to that size before feeding it into your network?
Your array with input data cannot have variable dimensions (this is a numpy limitation).
So the array, instead of being a regular array of numbers with 4 dimensions is being created as an array of arrays.
You should fit each image individually because of this limitation.
for epoch in range(epochs):
for img,class in zip(x,y):
#expand the first dimension of the image to have a batch size
img = img.reshape((1,) + img.shape)) #print and check there are 4 dimensions, like (1, width, height, 1).
class = class.reshape((1,) + class.shape)) #print and check there are two dimensions, like (1, classes).
model.train_on_batch(img,class,....)
I'm predicting image classes using Keras. It works in Google Cloud ML (GCML), but for efficiency need change it to pass base64 strings instead of json array. Related Documentation
I can easily run python code to decode a base64 string into json array, but when using GCML I don't have the opportunity to run a preprocessing step (unless maybe use a Lambda layer in Keras, but I don't think that is the correct approach).
Another answer suggested adding tf.placeholder with type of tf.string, which makes sense, but how to incorporate that into the Keras model?
Here is complete code for training the model and saving the exported model for GCML...
import os
import numpy as np
import tensorflow as tf
import keras
from keras import backend as K
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.preprocessing import image
from tensorflow.python.platform import gfile
IMAGE_HEIGHT = 138
IMAGE_WIDTH = 106
NUM_CLASSES = 329
def preprocess(filename):
# decode the image file starting from the filename
# end up with pixel values that are in the -1, 1 range
image_contents = tf.read_file(filename)
image = tf.image.decode_png(image_contents, channels=1)
image = tf.image.convert_image_dtype(image, dtype=tf.float32) # 0-1
image = tf.expand_dims(image, 0) # resize_bilinear needs batches
image = tf.image.resize_bilinear(image, [IMAGE_HEIGHT, IMAGE_WIDTH], align_corners=False)
image = tf.subtract(image, 0.5)
image = tf.multiply(image, 2.0) # -1 to 1
image = tf.squeeze(image,[0])
return image
filelist = gfile.ListDirectory("images")
sess = tf.Session()
with sess.as_default():
x = np.array([np.array( preprocess(os.path.join("images", filename)).eval() ) for filename in filelist])
input_shape = (IMAGE_HEIGHT, IMAGE_WIDTH, 1) # 1, because preprocessing made grayscale
# in our case the labels come from part of the filename
y = np.array([int(filename[filename.index('_')+1:-4]) for filename in filelist])
# convert class labels to numbers
y = keras.utils.to_categorical(y, NUM_CLASSES)
########## TODO: something here? ##########
image = K.placeholder(shape=(), dtype=tf.string)
decoded = tf.image.decode_jpeg(image, channels=3)
# scores = build_model(decoded)
model = Sequential()
# model.add(decoded)
model.add(Conv2D(32, kernel_size=(2, 2), activation='relu', input_shape=input_shape))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss=keras.losses.categorical_crossentropy,
optimizer=keras.optimizers.Adadelta(),
metrics=['accuracy'])
model.fit(
x,
y,
batch_size=64,
epochs=20,
verbose=1,
validation_split=0.2,
shuffle=False
)
predict_signature = tf.saved_model.signature_def_utils.build_signature_def(
inputs={'input_bytes':tf.saved_model.utils.build_tensor_info(model.input)},
########## TODO: something here? ##########
# inputs={'input': image }, # input name must have "_bytes" suffix to use base64.
outputs={'formId': tf.saved_model.utils.build_tensor_info(model.output)},
method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME
)
builder = tf.saved_model.builder.SavedModelBuilder("exported_model")
builder.add_meta_graph_and_variables(
sess=K.get_session(),
tags=[tf.saved_model.tag_constants.SERVING],
signature_def_map={
tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: predict_signature
},
legacy_init_op=tf.group(tf.tables_initializer(), name='legacy_init_op')
)
builder.save()
This is related to my previous question.
Update:
The heart of the question is how to incorporate the placeholder that calls decode into the Keras model. In other words, after creating the placeholder that decodes the base64 string to a tensor, how to incorporate that into what Keras runs? I assume it needs to be a layer.
image = K.placeholder(shape=(), dtype=tf.string)
decoded = tf.image.decode_jpeg(image, channels=3)
model = Sequential()
# Something like this, but this fails because it is a tensor, not a Keras layer. Possibly this is where a Lambda layer comes in?
model.add(decoded)
model.add(Conv2D(32, kernel_size=(2, 2), activation='relu', input_shape=input_shape))
...
Update 2:
Trying to use a lambda layer to accomplish this...
import keras
from keras.models import Sequential
from keras.layers import Lambda
from keras import backend as K
import tensorflow as tf
image = K.placeholder(shape=(), dtype=tf.string)
model = Sequential()
model.add(Lambda(lambda image: tf.image.decode_jpeg(image, channels=3), input_shape=() ))
Gives the error: TypeError: Input 'contents' of 'DecodeJpeg' Op has type float32 that does not match expected type of string.
first of all I use tf.keras but this should not be a big problem.
So here is an example of how you can read a base64 decoded jpeg:
def preprocess_and_decode(img_str, new_shape=[299,299]):
img = tf.io.decode_base64(img_str)
img = tf.image.decode_jpeg(img, channels=3)
img = tf.image.resize_images(img, new_shape, method=tf.image.ResizeMethod.BILINEAR, align_corners=False)
# if you need to squeeze your input range to [0,1] or [-1,1] do it here
return img
InputLayer = Input(shape = (1,),dtype="string")
OutputLayer = Lambda(lambda img : tf.map_fn(lambda im : preprocess_and_decode(im[0]), img, dtype="float32"))(InputLayer)
base64_model = tf.keras.Model(InputLayer,OutputLayer)
The code above creates a model that takes a jpeg of any size, resizes it to 299x299 and returns as 299x299x3 tensor. This model can be exported directly to saved_model and used for Cloud ML Engine serving. It is a little bit stupid, since the only thing it does is the convertion of base64 to tensor.
If you need to redirect the output of this model to the input of an existing trained and compiled model (e.g inception_v3) you have to do the following:
base64_input = base64_model.input
final_output = inception_v3(base64_model.output)
new_model = tf.keras.Model(base64_input,final_output)
This new_model can be saved. It takes base64 jpeg and returns classes identified by the inception_v3 part.
Another answer suggested adding tf.placeholder with type of tf.string, which makes sense, but how to incorporate that into the Keras model?
In Keras you can access your selected Backend (in this case Tensorflow) by doing:
from keras import backend as K
This you already seem to import on your code. That will enable you to access some native methods and resources available on the backend of your choice. It is the case that Keras backend includes a method for creating placeholders, among other utilities. Regarding placeholders, we can see what the Keras docs indicates about them:
placeholder
keras.backend.placeholder(shape=None, ndim=None, dtype=None, sparse=False, name=None)
Instantiates a placeholder tensor and returns it.
It also gives some example on its use:
>>> from keras import backend as K
>>> input_ph = K.placeholder(shape=(2, 4, 5))
>>> input_ph._keras_shape
(2, 4, 5)
>>> input_ph
<tf.Tensor 'Placeholder_4:0' shape=(2, 4, 5) dtype=float32>
As you can see, this is returning a Tensorflow tensor, with shape (2,4,5) and of dtype float. If you had another backend while doing the example you would get another tensor object (a Theano one surely). You can therefore use this placeholder() to adapt the solution you got on your previous question.
In conclusion, you can use your backend imported as K (or whatever you want) to do calls on the methods and objects available on the backend of your choice, by doing K.foo.bar() on the desired method. I suggest you give a read to what the Keras Backend to explore more things that can be useful for you on future situations.
Update: As per your edit. Yes, this placeholder should be a layer in your model. Specifically, it should be the Input Layer of your model, as it holds your decoded image (as Keras needs it that way) to classify.
I am trying to remove the top layers from a model I have previously trained.
This is the code I use:
import os
import h5py
import numpy as np
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Convolution2D, MaxPooling2D, ZeroPadding2D
from keras.layers import Activation, Dropout, Flatten, Dense
# KERAS_BACKEND=theano python
import keras
keras.backend.set_image_dim_ordering("th")
img_width, img_height = 150, 150
data_dir = '//shared_directory/projects/try_CD/data/validation'
nb_train_samples = 2000
nb_validation_samples = 800
nb_epoch = 50
def make_bottleneck_features(model):
datagen = ImageDataGenerator(rescale=1./255)
generator = datagen.flow_from_directory(
data_dir,
target_size=(img_width, img_height),
batch_size=32,
class_mode=None,
shuffle=False)
bottleneck_features = model.predict_generator(generator, nb_validation_samples)
return (bottleneck_features)
model=keras.models.load_model('/shared_directory/projects/think_exp/CD_M1.h5')
A = make_bottleneck_features(model)
model.summary()
for i in range (6):
model.pop()
B = make_bottleneck_features(model)
model.summary()
Judging comparing the results of the two calls to model.summary(), I can see that indeed the 6 topmost layers were removed.
However, the model's output (saved to A and B) does not change after discarding these layers.
What is the source of that discrepancy?
How can I retrieve the output of the desired layer instead of that of the entire model?
Thanks in advance!
You can't drop layers like that, in order for it to have an effect, you need to recompile the model (AKA model.compile).
But that's not the best way to obtain outputs from intermediate layers, you can just use K.function (where K is keras.backend) to build a function from the input to one of the layers and then call the function. More details are available in this answer.
Right now I am trying to construct a basic convolutional neural network to do simple classification with mnist dataset using keras. Eventually I want to put my own images in I just want to build a simple network first to make sure I have the structure working. so I downloaded the mnist data as mnint.pkl.gz unpacked it and loaded it into tuples and eventually bumpy arrays. Here is my code:
import numpy as np
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation, Flatten
from keras.layers.convolutional import Convolution2D, MaxPooling2D
from keras.optimizers import SGD
from PIL import Image as IM
import theano
from sklearn.cross_validation import train_test_split
import cPickle
import gzip
f=gzip.open('mnist.pkl.gz')
data1,data2,data3=cPickle.load(f)
f.close()
X=data1[0]
Y=data1[1]
x=X[0:15000,:]
y=Y[0:15000]
X_train,X_test,y_train,y_test=train_test_split(x,y,test_size
=0.33,random_state=99)
model=Sequential()
model.add(Convolution2D(10,5,5,border_mode='valid',
input_shape= (1,28,28)))
model.add(Activation('tanh'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Flatten())
model.add(Dense(10))
model.add(Activation('softmax'))
sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='mean_squared_error', optimizer=sgd)
model.fit(X_train,y_train, batch_size=10, nb_epoch=10)
score=model.evaluate(X_test,y_test,batch_size=10)
print(score)
I get an error as such:
'Wrong number of dimensions: expected 4, got 2 with shape
(10, 784).')
I think this means I need to put it into a theano 4d tensor such that
is has (samples,channels,rows,columns) but I have no idea how to do that. Furthermore when I specifically want to solve the problem I am after I will we loading '.png' files in, I was then going to put them into numpy matrices to feed in but it looks like that is not going to work. Can anybody tell me how I can get Images into theano4d tensors to use in this code? Thanks
You are correct that the code is expecting a tensor4. The conventional structure is (batch, channel, width, height). In this case the images are monochrome so channel=1 It looks like you're using a batch size of 10 and the MNIST images are 28 pixels in width and 28 pixels in height.
You can simply reshape the data into the format required. If x is of shape (10, 784) then x.reshape(10, 1, 28, 28) will have the required format.
The code is expecting a 4-dimensional numpy array, not a Theano tensor (keras does all the Theano tensor manipulation under the hood).
Your inputs, X_train and X_test need to be reshaped as follows:
X_train = X_train.reshape(-1, 1, 28, 28)
X_test = X_test.reshape(-1, 1, 28, 28)