I'm building an image classifier.
Had the output metrics that I wanted.
I added an ImageDataGenerator and the validation step added its own output metrics in the middle of the main output. Like So:
Epoch 58/300
10000/10000 [==============================] - 1s 65us/sample - loss: 0.7600 - acc: 0.8301
- 15s - loss: 0.5111 - acc: 0.9054 - val_loss: 0.7654 - val_acc: 0.8301
Here's the model compiler:
model.compile(
optimizer= SGD(
lr= 0.0,
momentum= hp['momentum'],
nesterov= True,
),
loss= categorical_crossentropy,
metrics= ['accuracy'],
)
Here's the ImageDataGenerator:
datagen = ImageDataGenerator(
width_shift_range=4,
height_shift_range=4,
fill_mode= 'constant',
cval = 0.0,
horizontal_flip=True,
)
Here's the fit function call before adding the generator:
model.fit(
x= train_data,
y= train_labels,
epochs= hp['num_epoch'],
verbose= 2,
callbacks= [lr_scheduler],
validation_data= (
valid_data, valid_labels
),
)
Epoch 8/300
- 21s - loss: 0.8771 - acc: 0.8077 - val_loss: 1.1955 - val_acc: 0.7258
And after the generator:
model.fit_generator(
datagen.flow(
train_data, train_labels,
batch_size= hp['batch_size'],
),
epochs= hp['num_epoch'],
verbose= 2,
callbacks= [lr_scheduler],
validation_data= (
valid_data, valid_labels),
)
Epoch 58/300
10000/10000 [==============================] - 1s 65us/sample - loss: 0.7600 - acc: 0.8301
- 15s - loss: 0.5111 - acc: 0.9054 - val_loss: 0.7654 - val_acc: 0.8301
Is this a known bug? Should I submit a bug report? Did I make a Stupid? Is there (hopefully) an easy solution?
Thanks, Any help would be appreciated.
I updated from TF 1.13.0 to TF 1.14.0 And it cleared up.
Apparently TensorFlow or Keras found this bug already.
Related
I want to load dataset from Kaggle. The link for the dataset is https://www.kaggle.com/sagyamthapa/handwritten-math-symbols.
It has images in different folder. How do I label the dataset and split and train it.
I did it the following way, but i got error
train_ds = tf.keras.preprocessing.image_dataset_from_directory(data_dir,#color_mode="grayscale",validation_split=0.2,subset="training",seed=123,image_size=(img_height, img_width),batch_size=batch_size)
val_ds = tf.keras.preprocessing.image_dataset_from_directory(data_dir,#color_mode="grayscale",validation_split=0.2,subset="validation",seed=123,image_size=(img_height, img_width),batch_size=batch_size)
code is shown below
import os
import pandas as pd
from sklearn.model_selection import train_test_split
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Dense, Activation,Dropout,Conv2D, MaxPooling2D,BatchNormalization
from tensorflow.keras.optimizers import Adam, Adamax
from tensorflow.keras.metrics import categorical_crossentropy
from tensorflow.keras import regularizers
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.models import Model, load_model, Sequential
sdir=r'../input/handwritten-math-symbols/dataset'
classlist=os.listdir(sdir)
filepaths=[]
labels=[]
classes=[]
for klass in classlist:
classpath=os.path.join(sdir, klass)
if os.path.isdir(classpath):
classes.append(klass)
flist=os.listdir(classpath)
for f in flist:
fpath=os.path.join(classpath,f)
if os.path.isfile(fpath):
filepaths.append(fpath)
labels.append(klass)
fseries=pd.Series(filepaths, name='filepaths')
Lseries=pd.Series (labels, name='labels')
df=pd.concat([fseries, Lseries], axis=1)
balance=df['labels'].value_counts()
print (balance) # dataset is reasonably balanced
train_split=.9
test_split=.05
dummy_split=test_split/(1-train_split)
train_df, dummy_df=train_test_split(df, train_size=train_split, shuffle=True, random_state = 123)
test_df, valid_df=train_test_split(dummy_df, train_size=dummy_split, shuffle=True, random_state=123)
def scalar(img):
return img/127.5-1 # scale pixels between -1 and + 1
gen=ImageDataGenerator(preprocessing_function=scalar)
train_gen=gen.flow_from_dataframe(train_df, x_col= 'filepaths', y_col='labels', target_size=(128,128), class_mode='categorical',
color_mode='rgb', shuffle=False)
test_gen=gen.flow_from_dataframe(test_df, x_col= 'filepaths', y_col='labels', target_size=(128,128), class_mode='categorical',
color_mode='rgb', shuffle=False)
valid_gen=gen.flow_from_dataframe(valid_df, x_col= 'filepaths', y_col='labels', target_size=(128,128), class_mode='categorical',
color_mode='rgb', shuffle=False)
base_model=tf.keras.applications.MobileNetV2( include_top=False, input_shape=(128,128,3), pooling='max', weights='imagenet')
x=base_model.output
x=keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001 )(x)
x = Dense(1024, kernel_regularizer = regularizers.l2(l = 0.016),activity_regularizer=regularizers.l1(0.006),
bias_regularizer=regularizers.l1(0.006) ,activation='relu', kernel_initializer= tf.keras.initializers.GlorotUniform(seed=123))(x)
x=Dropout(rate=.3, seed=123)(x)
output=Dense(len(classes), activation='softmax',kernel_initializer=tf.keras.initializers.GlorotUniform(seed=123))(x)
model=Model(inputs=base_model.input, outputs=output)
model.compile(Adamax(lr=.001), loss='categorical_crossentropy', metrics=['accuracy'])
estop=tf.keras.callbacks.EarlyStopping( monitor="val_loss", patience=4, verbose=1,restore_best_weights=True)
rlronp=tf.keras.callbacks.ReduceLROnPlateau( monitor="val_loss",factor=0.5, patience=1, verbose=1)
history=model.fit(x=train_gen, epochs=10, verbose=1, callbacks=[estop, rlronp], validation_data=valid_gen,
validation_steps=None, shuffle=False, initial_epoch=0)
save_path=r'c:\mydir\mymodel.h5' # specify the path to where to save model
model.save(save_path)
the results of model.fit should be as shown below
Epoch 1/20
254/254 [==============================] - 26s 84ms/step - loss: 14.9756 - accuracy: 0.8516 - val_loss: 5.0730 - val_accuracy: 0.6452
Epoch 2/20
254/254 [==============================] - 18s 73ms/step - loss: 2.7752 - accuracy: 0.9945 - val_loss: 1.7161 - val_accuracy: 0.7783
Epoch 3/20
254/254 [==============================] - 20s 78ms/step - loss: 0.7500 - accuracy: 0.9994 - val_loss: 0.9572 - val_accuracy: 0.8780
Epoch 4/20
254/254 [==============================] - 21s 84ms/step - loss: 0.3855 - accuracy: 0.9998 - val_loss: 0.6381 - val_accuracy: 0.9357
Epoch 5/20
254/254 [==============================] - 18s 71ms/step - loss: 0.2984 - accuracy: 1.0000 - val_loss: 0.4525 - val_accuracy: 0.9601
Epoch 6/20
254/254 [==============================] - 18s 73ms/step - loss: 0.2609 - accuracy: 1.0000 - val_loss: 0.3453 - val_accuracy: 0.9778
Epoch 7/20
254/254 [==============================] - 18s 70ms/step - loss: 0.2354 - accuracy: 0.9998 - val_loss: 0.2760 - val_accuracy: 0.9867
Epoch 8/20
254/254 [==============================] - 18s 69ms/step - loss: 0.2160 - accuracy: 1.0000 - val_loss: 0.2478 - val_accuracy: 0.9911
Epoch 9/20
254/254 [==============================] - 18s 70ms/step - loss: 0.2023 - accuracy: 1.0000 - val_loss: 0.2042 - val_accuracy: 0.9956
Epoch 10/20
254/254 [==============================] - 19s 74ms/step - loss: 0.1894 - accuracy: 1.0000 - val_loss: 0.1889 - val_accuracy: 0.9956
Is there any way to run tensorboard in google collab while using tensorflow-1.x? If not, how to use tensorboard in with tensorflow-1.x?
I would appreciate posting an any working example.
Yes, it is possible. Here is the complete working code to visualize histogram using Tensorboard in Google Colab.
%tensorflow_version 1.x
%load_ext tensorboard
import tensorflow as tf
print(tf.__version__)
import datetime, os
fashion_mnist = tf.keras.datasets.fashion_mnist
(x_train, y_train),(x_test, y_test) = fashion_mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
def create_model():
return tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(512, activation='relu'),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10, activation='softmax')
])
def train_model():
model = create_model()
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
logdir = os.path.join("logs", datetime.datetime.now().strftime("%Y%m%d-%H%M%S"))
tensorboard_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
model.fit(x=x_train,
y=y_train,
epochs=5,
validation_data=(x_test, y_test),
callbacks=[tensorboard_callback])
train_model()
%tensorboard --logdir logs
Output:
TensorFlow 1.x selected.
1.15.2
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz
32768/29515 [=================================] - 0s 0us/step
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz
26427392/26421880 [==============================] - 1s 0us/step
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz
8192/5148 [===============================================] - 0s 0us/step
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz
4423680/4422102 [==============================] - 0s 0us/step
WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
Train on 60000 samples, validate on 10000 samples
Epoch 1/5
60000/60000 [==============================] - 15s 250us/sample - loss: 0.4987 - acc: 0.8206 - val_loss: 0.4289 - val_acc: 0.8476
Epoch 2/5
60000/60000 [==============================] - 15s 253us/sample - loss: 0.3847 - acc: 0.8592 - val_loss: 0.3928 - val_acc: 0.8600
Epoch 3/5
60000/60000 [==============================] - 15s 246us/sample - loss: 0.3463 - acc: 0.8730 - val_loss: 0.3713 - val_acc: 0.8660
Epoch 4/5
60000/60000 [==============================] - 15s 246us/sample - loss: 0.3292 - acc: 0.8786 - val_loss: 0.3523 - val_acc: 0.8697
Epoch 5/5
60000/60000 [==============================] - 15s 249us/sample - loss: 0.3100 - acc: 0.8848 - val_loss: 0.3455 - val_acc: 0.8757
i have got error while fitting the elmo embedding model with training set of dimension x_tr=(43163, 50),and y_tr=
(43163, 50, 1) as :
InvalidArgumentError: Incompatible shapes: [1600] vs. [32,50]
[[{{node metrics/acc/Equal}} = Equal[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](metrics/acc/Reshape, metrics/acc/Cast)]].
how to solve this error ?
i tried to solve by making the training sample divisible by the batch size .
training set for fitting the model:
X_tr=np.array(X_tr)
print(X_tr.shape)
y_tr = np.array(y_tr).reshape(len(y_tr), max_len, 1)
print(y_tr.shape)
(43163, 50)
(43163, 50, 1)
making the model :
input_text = Input(shape=(max_len,), dtype=tf.string)
embedding = Lambda(ElmoEmbedding, output_shape=(None, 1024))(input_text)
x = Bidirectional(LSTM(units=512, return_sequences=True,
recurrent_dropout=0.2, dropout=0.2))(embedding)
x_rnn = Bidirectional(LSTM(units=512, return_sequences=True,
recurrent_dropout=0.2, dropout=0.2))(x)
x = add([x, x_rnn]) # residual connection to the first biLSTM
out = TimeDistributed(Dense(n_tags, activation="softmax"))(x)
model = Model(input_text, out)
compiling the model:
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"])
fitting the model:
fit_model = model.fit(np.array(X_tr), np.array(y_tr).reshape(len(y_tr), max_len, 1), validation_split=0.1,
batch_size=batch_size, epochs=5, verbose=1)
ERROR:
InvalidArgumentError: Incompatible shapes: [1600] vs. [32,50]
[[{{node metrics/acc/Equal}} = Equal[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](metrics/acc/Reshape, metrics/acc/Cast)]]
Expected result could be:
Train on 38816 samples, validate on 4320 samples
Epoch 1/5
38816/38816 [==============================] - 433s 11ms/step - loss: 0.0625 - acc: 0.9818 - val_loss: 0.0459 - val_acc: 0.9858
Epoch 2/5
38816/38816 [==============================] - 430s 11ms/step - loss: 0.0404 - acc: 0.9869 - val_loss: 0.0421 - val_acc: 0.9865
Epoch 3/5
38816/38816 [==============================] - 429s 11ms/step - loss: 0.0334 - acc: 0.9886 - val_loss: 0.0426 - val_acc: 0.9868
Epoch 4/5
38816/38816 [==============================] - 429s 11ms/step - loss: 0.0275 - acc: 0.9904 - val_loss: 0.0431 - val_acc: 0.9868
Epoch 5/5
38816/38816 [==============================] - 430s 11ms/step - loss: 0.0227 - acc: 0.9920 - val_loss: 0.0461 - val_acc: 0.9867
solved:
i have solved this issue by removing metrics=['accuracy']
but why this accuracy metrics game error i'm still unaware.
If anyone know it please help me out
I am currently working on a digit recognition challenge by Analytics Vidhya, the link to which is https://datahack.analyticsvidhya.com/contest/practice-problem-identify-the-digits/ .
The images in the dataset pertaining to this challenge are of dimensions 28*28*4 (28 = length = width , 4 = no. of channels).The code I have implemented is:
from keras.models import Sequential
from keras.layers import Dense,Dropout,Flatten,Activation
from keras.layers.convolutional import Conv2D
from keras.layers.convolutional import MaxPooling2D
from keras.utils import np_utils
from keras import backend as K
K.set_image_dim_ordering('th')
import numpy as np
# fix random seed for reproducibility
seed = 7
np.random.seed(seed)
# define the larger model
def larger_model():
# create model
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=(4, 28, 28),activation='relu',padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(15, (3, 3), activation='relu',padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(200, activation='relu'))
model.add(Dense(num_classes, activation='softmax'))
# Compile model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
def loadImages(path):
# return array of images
imagesList = listdir(path)
loadedImages = []
for image in imagesList:
img = io.imread(path + "/" + image,as_grey = False)
loadedImages.append(np.array(img))
return loadedImages
path = "C:/Users/Farz Jamal/Downloads/mnist/Train/Images/train" #path_to_train_dataset
import pandas as pd
df = pd.read_csv("C:/Users/Farz Jamal/Downloads/mnist/Train/train.csv") #path_to_class_labels
y = np.array(df['label'])
from sklearn.cross_validation import train_test_split as ttt
x_train,x_val,y_train,y_val = ttt(imgs,y,test_size = 0.2)
Continued Code:
x_vall,x_test,y_vall,y_test = ttt(x_val,y_val,test_size = 0.4)
x_train,x_vall,x_test = np.array(x_train).astype('float32'),np.array(x_vall).astype('float32'),np.array(x_test).astype('float32')
# normalize inputs from 0-255 to 0-1
x_train = x_train / 255.0
x_vall = x_vall / 255.0
x_test = x_test / 255.0
y_train = np_utils.to_categorical(y_train)
y_vall = np_utils.to_categorical(y_vall)
y_test = np_utils.to_categorical(y_test)
num_classes = y_vall.shape[1] #10
#fitting_and_evaluating
model = larger_model()
# Fit the model
model.fit(x_train, y_train, validation_data=(x_vall, y_vall), epochs=50, batch_size=200)
# Final evaluation of the model
scores = model.evaluate(x_test, y_test, verbose=0)
The output is coming as follows:(from 16thepoch to 37th epoch)
Epoch 16/50
39200/39200 [==============================] - 271s 7ms/step - loss: 2.3013 - acc: 0.1135 - val_loss: 2.3015 - val_acc: 0.1095
Epoch 17/50
39200/39200 [==============================] - 275s 7ms/step - loss: 2.3011 - acc: 0.1128 - val_loss: 2.3014 - val_acc: 0.1095
Epoch 18/50
39200/39200 [==============================] - 270s 7ms/step - loss: 2.3011 - acc: 0.1124 - val_loss: 2.3015 - val_acc: 0.1095
Epoch 19/50
39200/39200 [==============================] - 273s 7ms/step - loss: 2.3012 - acc: 0.1131 - val_loss: 2.3017 - val_acc: 0.1095
Epoch 20/50
39200/39200 [==============================] - 273s 7ms/step - loss: 2.3011 - acc: 0.1130 - val_loss: 2.3018 - val_acc: 0.1111
Epoch 21/50
39200/39200 [==============================] - 272s 7ms/step - loss: 2.3010 - acc: 0.1127 - val_loss: 2.3013 - val_acc: 0.1095
Epoch 22/50
39200/39200 [==============================] - 281s 7ms/step - loss: 2.3006 - acc: 0.1133 - val_loss: 2.3015 - val_acc: 0.1097
Epoch 23/50
39200/39200 [==============================] - 273s 7ms/step - loss: 2.3005 - acc: 0.1136 - val_loss: 2.3018 - val_acc: 0.1099
Epoch 24/50
39200/39200 [==============================] - 276s 7ms/step - loss: 2.3005 - acc: 0.1135 - val_loss: 2.3022 - val_acc: 0.1116
Epoch 25/50
39200/39200 [==============================] - 271s 7ms/step - loss: 2.2998 - acc: 0.1155 - val_loss: 2.3025 - val_acc: 0.1071
Epoch 26/50
39200/39200 [==============================] - 271s 7ms/step - loss: 2.2996 - acc: 0.1156 - val_loss: 2.3021 - val_acc: 0.1100
Epoch 27/50
39200/39200 [==============================] - 272s 7ms/step - loss: 2.2981 - acc: 0.1168 - val_loss: 2.3024 - val_acc: 0.1078
Epoch 28/50
39200/39200 [==============================] - 270s 7ms/step - loss: 2.2970 - acc: 0.1187 - val_loss: 2.3035 - val_acc: 0.1065
Epoch 29/50
39200/39200 [==============================] - 271s 7ms/step - loss: 2.2945 - acc: 0.1218 - val_loss: 2.3061 - val_acc: 0.1041
Epoch 30/50
39200/39200 [==============================] - 270s 7ms/step - loss: 2.2935 - acc: 0.1223 - val_loss: 2.3059 - val_acc: 0.1003
Epoch 31/50
39200/39200 [==============================] - 274s 7ms/step - loss: 2.2906 - acc: 0.1268 - val_loss: 2.3067 - val_acc: 0.1014
Epoch 32/50
39200/39200 [==============================] - 276s 7ms/step - loss: 2.2873 - acc: 0.1278 - val_loss: 2.3078 - val_acc: 0.1073
Epoch 33/50
39200/39200 [==============================] - 292s 7ms/step - loss: 2.2806 - acc: 0.1368 - val_loss: 2.3118 - val_acc: 0.1034
Epoch 34/50
39200/39200 [==============================] - 301s 8ms/step - loss: 2.2744 - acc: 0.1404 - val_loss: 2.3160 - val_acc: 0.1022
Epoch 35/50
39200/39200 [==============================] - 289s 7ms/step - loss: 2.2662 - acc: 0.1486 - val_loss: 2.3172 - val_acc: 0.1029
Epoch 36/50
39200/39200 [==============================] - 295s 8ms/step - loss: 2.2557 - acc: 0.1543 - val_loss: 2.3162 - val_acc: 0.1087
Epoch 37/50
39200/39200 [==============================] - 308s 8ms/step - loss: 2.2459 - acc: 0.1632 - val_loss: 2.3275 - val_acc: 0.1083
As can be seen, there is very low training as well validation accuracy.
I have tried reducing Dropout(previously it was 0.5 for one of the layers) but still no effect. I doubled the neurons in the last hidden layer,(previously they were 100), still no effect. It seems like, it is something to do with the pre processing of the images as well as the input parameters for the image.
What can be done?
Copied in from comments as the answer:
In fact your model isn't learning anything, which usually points to a bug. I don't see anything overtly wrong. A common error is inputting garbage to the network accidentally. Take the first few images that you're feeding to the network and display them in a debugger before your fit step and print out the labels and make sure they match. Do a sanity check on your inputs.
I am trying to find a useful code for improve classification using autoencoder.
I followed this example keras autoencoder vs PCA
But not for MNIST data, I tried to use it with cifar-10
so I made some changes but it seems like something is not fitting.
Could any one please help me in this?
if you have another example that can run in different dataset, that would help.
the validation in reduced.fit, which is (X_test,Y_test) is not learned, so it gives wronf accuracy in .evalute()
always give
val_loss: 2.3026 - val_acc: 0.1000
This is the code, and the error:
rom keras.datasets import cifar10
from keras.models import Model
from keras.layers import Input, Dense
from keras.utils import np_utils
import numpy as np
num_train = 50000
num_test = 10000
height, width, depth = 32, 32, 3 # MNIST images are 28x28
num_classes = 10 # there are 10 classes (1 per digit)
(X_train, y_train), (X_test, y_test) = cifar10.load_data()
X_train = X_train.reshape(num_train,height * width * depth)
X_test = X_test.reshape(num_test,height * width*depth)
X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
X_train /= 255 # Normalise data to [0, 1] range
X_test /= 255 # Normalise data to [0, 1] range
Y_train = np_utils.to_categorical(y_train, num_classes) # One-hot encode the labels
Y_test = np_utils.to_categorical(y_test, num_classes) # One-hot encode the labels
input_img = Input(shape=(height * width * depth,))
s=height * width * depth
x = Dense(s, activation='relu')(input_img)
encoded = Dense(s//2, activation='relu')(x)
encoded = Dense(s//8, activation='relu')(encoded)
y = Dense(s//256, activation='relu')(x)
decoded = Dense(s//8, activation='relu')(y)
decoded = Dense(s//2, activation='relu')(decoded)
z = Dense(s, activation='sigmoid')(decoded)
model = Model(input_img, z)
model.compile(optimizer='adadelta', loss='mse') # reporting the accuracy
model.fit(X_train, X_train,
nb_epoch=10,
batch_size=128,
shuffle=True,
validation_data=(X_test, X_test))
mid = Model(input_img, y)
reduced_representation =mid.predict(X_test)
out = Dense(num_classes, activation='softmax')(y)
reduced = Model(input_img, out)
reduced.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
reduced.fit(X_train, Y_train,
nb_epoch=10,
batch_size=128,
shuffle=True,
validation_data=(X_test, Y_test))
scores = reduced.evaluate(X_test, Y_test, verbose=0)
print("Accuracy: ", scores[1])
Train on 50000 samples, validate on 10000 samples
Epoch 1/10
50000/50000 [==============================] - 5s - loss: 0.0639 - val_loss: 0.0633
Epoch 2/10
50000/50000 [==============================] - 5s - loss: 0.0610 - val_loss: 0.0568
Epoch 3/10
50000/50000 [==============================] - 5s - loss: 0.0565 - val_loss: 0.0558
Epoch 4/10
50000/50000 [==============================] - 5s - loss: 0.0557 - val_loss: 0.0545
Epoch 5/10
50000/50000 [==============================] - 5s - loss: 0.0536 - val_loss: 0.0518
Epoch 6/10
50000/50000 [==============================] - 5s - loss: 0.0502 - val_loss: 0.0461
Epoch 7/10
50000/50000 [==============================] - 5s - loss: 0.0443 - val_loss: 0.0412
Epoch 8/10
50000/50000 [==============================] - 5s - loss: 0.0411 - val_loss: 0.0397
Epoch 9/10
50000/50000 [==============================] - 5s - loss: 0.0391 - val_loss: 0.0371
Epoch 10/10
50000/50000 [==============================] - 5s - loss: 0.0377 - val_loss: 0.0403
Train on 50000 samples, validate on 10000 samples
Epoch 1/10
50000/50000 [==============================] - 3s - loss: 2.3605 - acc: 0.0977 - val_loss: 2.3026 - val_acc: 0.1000
Epoch 2/10
50000/50000 [==============================] - 3s - loss: 2.3027 - acc: 0.0952 - val_loss: 2.3026 - val_acc: 0.1000
Epoch 3/10
50000/50000 [==============================] - 3s - loss: 2.3027 - acc: 0.0978 - val_loss: 2.3026 - val_acc: 0.1000
Epoch 4/10
50000/50000 [==============================] - 3s - loss: 2.3027 - acc: 0.0980 - val_loss: 2.3026 - val_acc: 0.1000
Epoch 5/10
50000/50000 [==============================] - 3s - loss: 2.3027 - acc: 0.0974 - val_loss: 2.3026 - val_acc: 0.1000
Epoch 6/10
50000/50000 [==============================] - 3s - loss: 2.3027 - acc: 0.1000 - val_loss: 2.3026 - val_acc: 0.1000
Epoch 7/10
50000/50000 [==============================] - 3s - loss: 2.3027 - acc: 0.0992 - val_loss: 2.3026 - val_acc: 0.1000
Epoch 8/10
50000/50000 [==============================] - 3s - loss: 2.3027 - acc: 0.0982 - val_loss: 2.3026 - val_acc: 0.1000
Epoch 9/10
50000/50000 [==============================] - 3s - loss: 2.3027 - acc: 0.0965 - val_loss: 2.3026 - val_acc: 0.1000
Epoch 10/10
50000/50000 [==============================] - 3s - loss: 2.3027 - acc: 0.0978 - val_loss: 2.3026 - val_acc: 0.1000
9856/10000 [============================>.] - ETA: 0s('Accuracy: ', 0.10000000000000001)
there are multiple issues with your code.
Your autoencoder is not fully trained, if you plot the training data, you will see the model haven't converged yet. By
history = model.fit(X_train, X_train,
nb_epoch=10,
batch_size=128,
shuffle=True,
validation_data=(X_test, X_test))
you will obtain the loss values during training. If you plot them, e.g. in matplotlib,
import matplotlib.pyplot as plt
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model train vs validation loss 1')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'validation'], loc='upper right')
plt.show()
you will see that it needs more epochs to converge.
The autoencoder architecture is wrongly built, there is typo in line y = Dense(s//256, activation='relu')(x), you probably wanted to usey = Dense(s//256, activation='linear')(encoded) so it uses previous layer and not the input. And also you don't want to use the relu activation in latent space, because then it disallows you subtracting latent variables from each other and thus makes the autoencoder much less efficient.
With those fixes, the model trains withour problems.
I increased number of epochs to 30 for training both networks so it will train better.
At the end of the trainings, the classification model reports loss: 1.2881 - acc: 0.5397 - val_loss: 1.3841 - val_acc: 0.5126 which is lower than you experienced.