image_dim_ordering - what am I missing here? - keras

Question

EDIT: Could not reproduce this issue using cuda 8.0 and using titan X (Pascal)
Using tensorflow backend for keras I have issues that are related to image_dim_ordering.
When I use image_dim_ordering='th' in the keras config file, everything works well But when I use 'tf', training simply doesn't really improve much from 0.5 accuracy.
The motivation is that currently my live augmentations are very costly, and I'd love to remove the unneeded reshape from theano dim order convention to tensorflow.
I tried recreating the issue with simple code to allow reproduction by other people which may assist me to understand what am I doing wrong here. I'm well aware of the channel,height,width different conventions, and at least I think that I handle that.
While I didn't fully reproduce my problem in the compact example (maybe because it's a trivial task), the training results are repeatedly different, and much worse for the 'tf' case, even when I try different seed values.
Note - in this reproducing code, all that the network needs to do is to tell apart full patches of -1.0 from full patches of 1.0
This is my '~/.keras/keras.json'
{
"floatx": "float32",
"epsilon": 1e-07,
"backend": "tensorflow",
"image_dim_ordering": "th"
}
my tensorflow version is ''0.11.0rc0'' (it happened on 0,10 as well)
my keras is latest git pull of today.
Using 'th' for the image_dim_ordering I get accuracy >=0.99 at epoch 4 for three different seeds.
Using 'tf' for the dim order I get accuracy >= 0.9 much latest as you can see below in the log, only at around epoch 24
The following is a standalone code that should reproduce the problem:
from keras import backend as K
import keras.optimizers
from keras.layers import Convolution2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense, Input
from keras.models import Model
import numpy as np
def make_model(input_dim_size):
if K.image_dim_ordering() == 'tf':
input_shape = (input_dim_size, input_dim_size,1)
else:
input_shape = (1, input_dim_size, input_dim_size)
img_input = Input(shape=input_shape)
x = Convolution2D(64,5,5,border_mode='same')(img_input)
x = Activation('relu')(x)
x = MaxPooling2D((2,2),strides=(2,2))(x)
x = Convolution2D(64, 5, 5, border_mode='same')(x)
x = Activation('relu')(x)
x = MaxPooling2D((2, 2), strides=(2, 2))(x)
x = Convolution2D(64, 5, 5, border_mode='same')(x)
x = Activation('relu')(x)
x = MaxPooling2D((2, 2), strides=(2, 2))(x)
x = Convolution2D(128, 5, 5, border_mode='same')(x)
x = Activation('relu')(x)
x = MaxPooling2D((2, 2), strides=(2, 2))(x)
x = Convolution2D(128, 5, 5, border_mode='same')(x)
x = Activation('relu')(x)
x = MaxPooling2D((2, 2), strides=(2, 2))(x)
x = Flatten()(x)
x = Dense(1024*2)(x)
x = Activation('relu')(x)
x = Dropout(0.5)(x)
x = Dense(1024 * 2)(x)
x = Activation('relu')(x)
x = Dropout(0.75)(x)
x = Dense(200)(x)
x = Activation('relu')(x)
x = Dropout(0.75)(x)
x = Dense(1,activation='sigmoid')(x)
model = Model(img_input, x)
learning_rate = 0.01
sgd = keras.optimizers.sgd(lr=learning_rate, momentum=0.9, nesterov=True)
model.summary()
model.compile(loss='binary_crossentropy',
optimizer=sgd,
metrics=['accuracy']
)
return model
np.random.seed(456)
def dummy_generator(mini_batch_size=64, block_size=100):
if K.image_dim_ordering() == 'tf':
tensor_X_shape = (mini_batch_size,block_size, block_size,1)
else:
tensor_X_shape = (mini_batch_size, 1, block_size, block_size)
X = np.zeros(tensor_X_shape, dtype=np.float32)
y = np.zeros((mini_batch_size, 1))
while True:
for b in range(mini_batch_size):
X[b, :, :, :] = (float(b % 2) * 2.0) - 1.0
y[b, :] = float(b % 2)
yield X,y
with K.tf.device('/gpu:2'):
K.set_session(K.tf.Session(config=K.tf.ConfigProto(allow_soft_placement=True, log_device_placement=False)))
MINI_BATCH_SIZE = 64
PATCH_SIZE = 100
model = make_model(PATCH_SIZE)
gen = dummy_generator(mini_batch_size=MINI_BATCH_SIZE,block_size=PATCH_SIZE)
model.fit_generator(gen, MINI_BATCH_SIZE*10,
100, verbose=1,
callbacks=[],
validation_data=None,
nb_val_samples=None,
max_q_size=1,
nb_worker=1, pickle_safe=False)
For the 'tf' case this is the training log: (and looks very similar on different seeds):
Epoch 1/100
640/640 [==============================] - 1s - loss: 0.6932 - acc: 0.4781
Epoch 2/100
640/640 [==============================] - 0s - loss: 0.6932 - acc: 0.4938
Epoch 3/100
640/640 [==============================] - 0s - loss: 0.6921 - acc: 0.5203
Epoch 4/100
640/640 [==============================] - 0s - loss: 0.6920 - acc: 0.5469
Epoch 5/100
640/640 [==============================] - 0s - loss: 0.6935 - acc: 0.4875
Epoch 6/100
640/640 [==============================] - 0s - loss: 0.6941 - acc: 0.4969
Epoch 7/100
640/640 [==============================] - 0s - loss: 0.6937 - acc: 0.5047
Epoch 8/100
640/640 [==============================] - 0s - loss: 0.6931 - acc: 0.5312
Epoch 9/100
640/640 [==============================] - 0s - loss: 0.6923 - acc: 0.5250
Epoch 10/100
640/640 [==============================] - 0s - loss: 0.6929 - acc: 0.5281
Epoch 11/100
640/640 [==============================] - 0s - loss: 0.6934 - acc: 0.4953
Epoch 12/100
640/640 [==============================] - 0s - loss: 0.6918 - acc: 0.5234
Epoch 13/100
640/640 [==============================] - 0s - loss: 0.6930 - acc: 0.5125
Epoch 14/100
640/640 [==============================] - 0s - loss: 0.6939 - acc: 0.4797
Epoch 15/100
640/640 [==============================] - 0s - loss: 0.6936 - acc: 0.5047
Epoch 16/100
640/640 [==============================] - 0s - loss: 0.6917 - acc: 0.4922
Epoch 17/100
640/640 [==============================] - 0s - loss: 0.6945 - acc: 0.4891
Epoch 18/100
640/640 [==============================] - 0s - loss: 0.6948 - acc: 0.5000
Epoch 19/100
640/640 [==============================] - 0s - loss: 0.6968 - acc: 0.4594
Epoch 20/100
640/640 [==============================] - 0s - loss: 0.6919 - acc: 0.5391
Epoch 21/100
640/640 [==============================] - 0s - loss: 0.6904 - acc: 0.5172
Epoch 22/100
640/640 [==============================] - 0s - loss: 0.6881 - acc: 0.5906
Epoch 23/100
640/640 [==============================] - 0s - loss: 0.6804 - acc: 0.6359
Epoch 24/100
640/640 [==============================] - 0s - loss: 0.6470 - acc: 0.8219
Epoch 25/100
640/640 [==============================] - 0s - loss: 0.4134 - acc: 0.9625
Epoch 26/100
640/640 [==============================] - 0s - loss: 0.2347 - acc: 0.9953
Epoch 27/100
640/640 [==============================] - 0s - loss: 0.1231 - acc: 1.0000
And for the 'th' case the training log is (and looks very similar on different seeds):
Epoch 1/100
640/640 [==============================] - 3s - loss: 0.6891 - acc: 0.5594
Epoch 2/100
640/640 [==============================] - 2s - loss: 0.6079 - acc: 0.7328
Epoch 3/100
640/640 [==============================] - 2s - loss: 0.3166 - acc: 0.9422
Epoch 4/100
640/640 [==============================] - 2s - loss: 0.1767 - acc: 0.9969
I find it suspicious that it's so fast in the tensorflow case, (0s), but after adding debug printing to the generator it does seem to get called.
I thought that maybe it's related to keras not needing to reshape anything, but 2-3 seconds sounds too much time for this amount of reshapes
If anyone can try to reproduce the results that I see and help me understand what the heck am I missing here, I'd be grateful :)

Answer 1

This thread is bit old but I am still replying in case someone faces the same issues.
The error is caused due to the inconsistent Keras backend configuration...
{
"floatx": "float32",
"epsilon": 1e-07,
"backend": "tensorflow",
"image_dim_ordering": "th"
}
The configuration uses tensorflow as backend but uses image dimension ordering of Theano instead of tensorflow. change image_dim_ordering to tf and that should solve the issue..
"image_dim_ordering": "tf"