Order of rotated images by using a custom generator - python-3.x

Question

I use a custom image data generator for my project. It receives batches of images and returns [0, 90, 180 and 270] degrees rotated versions of the images with the corresponding class indices {0:0, 1:90, 2:180, 3:270}. Lets assume we have images A, B and C in a batch and images A to Z in the whole data set. All the images are naturally in 0 degree orientation. Initially I returned all the rotated images at the same time. Here is a sample of returned batch: [A0,B0,C0,A1,B1,C1,...,A3,B3,C3]. But this gave me useless results. To compare my approach I trained the same model by using my generator and built in Keras ImageDataGenerator with flow_from_directory. For the built in function I manually rotated original images and stored them in separate folders. Here are the accuracy plots for comparison:
I used only a few images just to see if there is any difference. From the plots it is obvious that the custom generator is not correct. Hence I think it must return the images as [[A0,B0,C0],[D0,E0,F0]...[...,Z0]], then [[A1,B1,C1],[D1,E1,F1]...[...,Z1]] and so on. To do this I must use the folowing function for multiple times (in my case 4).
def next(self):
with self.lock:
# get input data index and size of the current batch
index_array = next(self.index_generator)
# create array to hold the images
return self._get_batches_of_transformed_samples(index_array)
This function iterates through the directory and returns batches of images. When it reaches to the last image it finishes and the next epoch starts. In my case, in one epoch I want to run this for 4 times by sending the rotation angle as an argument like this: self._get_batches_of_transformed_samples(index_array) , rotation_angle). I was wondering if this is possible or not? If not what could be the solution? Here is the current data generator code:
def _get_batches_of_transformed_samples(self, index_array):
# create list to hold the images and labels
batch_x = []
batch_y = []
# create angle categories corresponding to number of rotation angles
angle_categories = list(range(0, len(self.target_angles)))
# generate rotated images and corresponding labels
for rotation_angle, angle_indice in zip(self.target_angles, angle_categories):
for i, j in enumerate(index_array):
if self.filenames is None:
image = self.images[j]
if len(image.shape) == 2: image = cv2.cvtColor(image,cv2.COLOR_GRAY2RGB)
else:
is_color = int(self.color_mode == 'rgb')
image = cv2.imread(self.filenames[j], is_color)
if is_color:
if not image is None:
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# do nothing if the image is none
if not image is None:
rotated_im = rotate(image, rotation_angle, self.target_size[:2])
if self.preprocess_func: rotated_im = self.preprocess_func(rotated_im)
# add dimension to account for the channels if the image is greyscale
if rotated_im.ndim == 2: rotated_im = np.expand_dims(rotated_im, axis=2)
batch_x.append(rotated_im)
batch_y.append(angle_indice)
# convert lists to numpy arrays
batch_x = np.asarray(batch_x)
batch_y = np.asarray(batch_y)
batch_y = to_categorical(batch_y, len(self.target_angles))
return batch_x, batch_y
def next(self):
with self.lock:
# get input data index and size of the current batch
index_array = next(self.index_generator)
# create array to hold the images
return self._get_batches_of_transformed_samples(index_array)

Answer 1

Hmm I would probably do this through keras.utils.Sequence
from keras.utils import Sequence
import numpy as np
class RotationSequence(Sequence):
def __init__(self, x_set, y_set, batch_size, rotations=(0,90,180,270)):
self.rotations = rotations
self.x, self.y = x_set, y_set
self.batch_size = batch_size
def __len__(self):
return int(np.ceil(len(self.x) / float(self.batch_size)))
def __getitem__(self, idx):
batch_x = self.x[idx * self.batch_size:(idx + 1) * self.batch_size]
batch_y = self.y[idx * self.batch_size:(idx + 1) * self.batch_size]
x, y = [], []
for rot in self.rotations:
x += [rotate(cv2.imread(file_name), rotation_angle) for file_name in batch_x]
y += batch_y
return np.array(x), np.array(y)
def on_epoch_end(self):
shuffle_idx = np.random.permutation(len(self.x))
self.x, self.y = self.x[shuffle_idx], self.y[shuffle_idx]
And then just pass the batcher to model.fit()
rotation_batcher = RotationSequence(...)
model.fit_generator(rotation_batcher,
steps_per_epoch=len(rotation_batcher),
validation_data=validation_batcher,
epochs=epochs)
This allows you to have more control over the batches being fed into your model. This implementation will almost run. You just need to implement the rotate() function in __getitem__. Also, the batch_size will be 4 times the set size because I just duplicated and rotated each batch. Hope this is helpful to you