Is there any way to combine two models GRU and DropBlock? - keras

Combine two models DropBlock(dimesnions=4), and CuDNNGRU(dimension=3) for MNIST image Classification. How can I combine different dimensions of Drop Block and GRU models?
GRU mostly used for Document classification but here I'm trying to classify MNIST dataset. Both Models separately score ~98%. I want to combine both DropBlock and GRU Model in a single classification Model.
The codes of combining the DropBlock and GRU model for MNIST classification.
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train/255.0
x_test = x_test/255.0
print(x_train.shape)
print(x_train[0].shape)
#load the model
model = Sequential()
gru = Sequential()
gru.add(CuDNNGRU(128,input_shape=(x_train.shape[1:]),
return_sequences=True))
gru.add(BatchNormalization())
gru.add(CuDNNGRU(128))
gru.add(BatchNormalization())
print(gru.output_shape)
drop = Sequential()
drop.add(DropBlock2D(input_shape=(28, 28, 1), block_size=7,
keep_prob=0.8, name='Input-Dropout'))
drop.add(Conv2D(filters=64, kernel_size=3, activation='relu',
padding='same', name='Conv-1'))
drop.add(MaxPool2D(pool_size=2, name='Pool-1'))
drop.add(DropBlock2D(block_size=5, keep_prob=0.8, name='Dropout-1'))
drop.add(Conv2D(filters=32, kernel_size=3, activation='relu',
padding='same', name='Conv-2'))
drop.add(MaxPool2D(pool_size=2, name='Pool-2'))
print(drop.output_shape)
model.add(merge([drop, gru], mode='concat', concat_axis=2))
print(model.output_shape)
model.add(Flatten(name='Flatten'))
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(10, activation='softmax'))
score = model.evaluate(x_test, y_test, verbose=0)
The expected result ~98% results on this classification model.

Related

ValueError: input tensor must have rank 4 TensorFlow

i am using fashion_mnist images database (60,000 small square 28×28 pixel grayscale images) and i am trying to apply CNN-LSTM in cascading, this is the code i am using:
from tensorflow.keras.datasets import fashion_mnist
(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()
print("Shape of x_train: {}".format(x_train.shape))
print("Shape of y_train: {}".format(y_train.shape))
print()
print("Shape of x_test: {}".format(x_test.shape))
print("Shape of y_test: {}".format(y_test.shape))
# define CNN model
model = Sequential()
model.add(TimeDistributed(Conv2D(32, kernel_size=(3, 3),
activation='relu',
input_shape=(60000,28,28))))
model.add(TimeDistributed(Conv2D(64, (3, 3), activation='relu')))
model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))
model.add(TimeDistributed((Dropout(0.25))))
model.add(TimeDistributed(Flatten()))
## LSTM
model.add(LSTM(200, activation='relu', return_sequences=True))
model.add(Dense(128, activation='relu'))
model.compile(loss='categorical_crossentropy', optimizer='adam',
metrics=['accuracy'])
##fitting model
model.fit(x_train,y_train,epochs=5)
test_loss, test_acc=model.evaluate(x_test,y_test)
print('Loss: {0}-Acc:{1}')
print(test_acc)
i get the error after running the fitting line, can any one help me solving the error.
Define an Input layer with a four-channel output instead of defining the input with Conv2D layer.

Difference between keras.evaluate() accuracy e manually calculated accuracy by prediction: Multilabel classification

I'm using keras to perform multilabel classification. I'm using 'binary_crossentropy' as loss function, metrics=['accuracy'], 'sigmoid' as activation.
During the training I saw accuracy above the 90%, and also using evaluate on the test set I have something similar.
If I try to manually compute the accuracy using predict module the accuracy also on the training set dramatically leaves 45%.
This is the model:
model = keras.models.Sequential()
model.add(keras.layers.Conv2D(8, kernel_size=3, strides=2, activation='relu', input_shape=(N_qubits, N_qubits,2)))
model.add(keras.layers.Conv2D(16, kernel_size=2, activation='relu'))
model.add(keras.layers.Dense(64, activation='relu'))
model.add(keras.layers.Dense(128, activation='relu'))
model.add(keras.layers.Conv2D(32, kernel_size=1, activation='relu'))
model.add(keras.layers.GlobalMaxPooling2D())
model.add(keras.layers.Dense(units=y_train.shape[1], activation='sigmoid'))
adam_optimizer = keras.optimizers.adam()
model.compile(loss='binary_crossentropy', optimizer=adam_optimizer, metrics=['accuracy'])
history = model.fit(X_train, y_train, batch_size=150, epochs=1000, verbose=1, validation_split=0.05)
Here where I use evaluate()
results = model.evaluate(X_test, y_test, batch_size=128)
Here when I use predict()
y_pred_train = model.predict(X_train)
y_pred_test = model.predict(X_test)

LSTM Grid Search

I have a code below which implements an architecture (in grid search), to yield appropriate parameters for input, nodes, epochs, batch size and differenced time series input.
The challenge I have is to convert the neural network from just having one LSTM hidden layer, to multiple LSTM hidden layers.
At the moment, I could only run the code with Dense-type hidden layers, without having any errors thrown, otherwise I get dimension errors, tuple errors and so on.
The problem is only persistent in the neural network architecture section.
Original code that works:
def model_fit(train, config):
# unpack config
n_input, n_nodes, n_epochs, n_batch, n_diff = config
# Data
if n_diff > 0:
train = difference(train, n_diff)
# Time series to supervised format
data = series_to_supervised(train, n_in=n_input)
train_x, train_y = data[:, :-1], data[:, -1]
# Reshaping input data into [samples, timesteps, features]
n_features = 1
train_x = train_x.reshape((train_x.shape[0], train_x.shape[1], n_features))
# Define model for (Grid search architecture)
model = Sequential()
model.add(LSTM(n_nodes, activation='relu', input_shape=(n_input, n_features)))
model.add(Dense(n_nodes, activation='relu'))
model.add(Dense(n_nodes, activation='relu'))
model.add(Dense(n_nodes, activation='relu'))
model.add(Dense(1))
# Compile model (Grid search architecture)
model.compile(loss='mse', optimizer='adam')
# fit model
model.fit(train_x, train_y, epochs=n_epochs, batch_size=n_batch, verbose=0)
return model
Modified LSTM-hidden layer code, that fails to run:
# Define model for (Grid search architecture)
model = Sequential()
model.add(LSTM(n_nodes, activation='relu', input_shape=(n_input, n_features), return_sequences=True))
model.add(LSTM(n_nodes, activation='relu', return_sequences=True))
model.add(LSTM(n_nodes, activation='relu', return_sequences=True))
model.add(LSTM(n_nodes, activation='relu', return_sequences=True))
model.add(TimeDistributed(Dense(1)))
Another variant that also threw an error - ValueError: Error when checking target: expected time_distributed_4 to have 3 dimensions, but got array with shape (34844, 1)
model = Sequential()
model.add(LSTM(n_nodes, activation='relu', input_shape=(n_input, n_features), return_sequences=True))
model.add(LSTM(n_nodes, activation='relu', return_sequences=False))
model.add(RepeatVector(n_input))
model.add(LSTM(n_nodes, activation='relu', return_sequences=True))
model.add(LSTM(n_nodes, activation='relu', return_sequences=True))
model.add(TimeDistributed(Dense(n_features)))
Could anyone with any suggestion please help me ?
Try to set return_sequences=False at the last layer.

Error when checking target: expected dense to have shape (1,) but got array with shape (15662,) maxpooling as a first layer

I'm trying to use maxpooling as a first layer using keras and I have a problem with the input and output dimensions.
print(x_train.shape)
print(y_train.shape)
(15662, 6)
(15662,)
x_train = np.reshape(x_train, (-1,15662, 6))
y_train = label_array.reshape(1, -1)
model = Sequential()
model.add(MaxPooling1D(pool_size = 2 , strides=1, input_shape = (15662,6)))
model.add(Dense(5, activation='relu'))
model.add(Flatten())
model.add(Dense(1, activation='softmax'))
model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=
['accuracy'])
model.fit(x_train, y_train, batch_size= 32, epochs=1)
After running the model, I get the following error:
ValueError: Error when checking target: expected dense_622 (last layer)
to have shape (1,) but got array with shape (15662,)
I'm doing classification and my target is binary (0,1)
Thank you
Your target should have shape (batch_size, 1) but you are passing an array of shape (1, 15662). It seems like 15662 should be the batch size, in which case x_train should have shape (15662, 6) and y_train should have shape (15662, 1). In this case however, it doesn't make any sense to have a MaxPooling1D layer as the first layer of your model since max pooling requires a 3D input (i.e. shape (batch_size, time_steps, features)). You probably want to leave out the max pooling layer (and the Flatten layer). The following code should work:
# x_train: (15662, 6)
# y_train: (15662,)
model = Sequential()
model.add(Dense(5, activation='relu', input_shape=(6,))) # Note: don't specify the batch size in input_shape
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=
['accuracy'])
model.fit(x_train, y_train, batch_size= 32, epochs=1)
But it of course depends on what kind of data you have.

How can I train on video data using Keras? "transfer learning"

I want to train my model on video data for gesture recognition, proposed using LSTM's and TimeDistributed layers. Would this be ideal way to tackle my problem?
# Convolution
pool_size = 4
# LSTM
lstm_output_size = 1
print('Build model...')
model = Sequential()
model.add(TimeDistributed(Dense(62), input_shape=(img_width, img_height,3)))
model.add(Conv2D(32, (3, 3)))
model.add(Dropout(0.25))
model.add(Conv2D(32, (3, 3)))
model.add(MaxPooling2D(pool_size=pool_size))
# model.add(Dense(1))
model.add(TimeDistributed(Flatten()))
model.add(CuDNNLSTM(256, return_sequences=True))
model.add(CuDNNLSTM(256, return_sequences=True))
model.add(CuDNNLSTM(256, return_sequences=True))
model.add(CuDNNLSTM(lstm_output_size))
model.add(Dense(units = 1, activation = 'sigmoid'))
print('Train...')
model.summary()
# Run epochs of sampling data then training
For temporal sequence data LSTM networks are generally the right choice. If you want to analyze video then a combination with 2d convolutions sounds reasonable to me. However, you have to apply TimeDistributed on all layers which dont expect sequence data. In your example that means all laysers expect LSTM.
# Convolution
pool_size = 4
# LSTM
lstm_output_size = 1
print('Build model...')
model = Sequential()
model.add(TimeDistributed(Dense(62), input_shape=(img_width, img_height,3)))
model.add(TimeDistributed(Conv2D(32, (3, 3))))
model.add(Dropout(0.25))
model.add(TimeDistributed(Conv2D(32, (3, 3))))
model.add(TimeDistributed(MaxPooling2D(pool_size=pool_size)))
# model.add(Dense(1))
model.add(TimeDistributed(Flatten()))
model.add(CuDNNLSTM(256, return_sequences=True))
model.add(CuDNNLSTM(256, return_sequences=True))
model.add(CuDNNLSTM(256, return_sequences=True))
model.add(CuDNNLSTM(lstm_output_size))
model.add(Dense(units = 1, activation = 'sigmoid'))
print('Train...')
model.summary()
# run epochs of sampling data then training
The last Dense Layer can stay this way because the final lstm doesnt output a sequence.

Resources