Keras stateful LSTM error - keras

Question

I want to create stateful LSTM in keras. I gave it a command like this:
model.add(LSTM(300,input_dim=4,activation='tanh',stateful=True,batch_input_shape=(19,13,4),return_sequences=True))
where batch size=19. But on running it gives error
Exception: In a stateful network, you should only pass inputs with a number of samples that can be divided by the batch size. Found: 8816 samples. Batch size: 32.
I did not specify batch size 32 anywhere in my script and 19 is divisible by 8816

Answer 1

model.fit() does the batching (as opposed to model.train_on_batch for example). Consequently it has a batch_size parameter which defaults to 32.
Change this to your input batch size and it should work as expected.
Example:
batch_size = 19
model = Sequential()
model.add(LSTM(300,input_dim=4,activation='tanh',stateful=True,batch_input_shape=(19,13,4),return_sequences=True))
model.fit(x, y, batch_size=batch_size)

Answer 2

there are two cases where batch_size error could occur.
model.fit(train_x, train_y, batch_size= n_batch, shuffle=True,verbose=2)
trainPredict = model.predict(train_x, batch_size=n_batch)
or testPredict = model.predict(test_x,batch_size=n_batch)
in both cases, you have to mention no. of batches.
note: we need to predict train and test both, so best practice is divide the test and train in such a way that your batch size is multiple of both in stateful=True case

Answer 3

To dynamically size your data and batches:
Size data and training sample split:
data_size = int(len(supervised_values))
train_size_initial = int(data_size * train_split)
x_samples = supervised_values[-data_size:, :]
Size number of training samples to batch size:
if train_size_initial < batch_size_div:
batch_size = 1
else:
batch_size = int(train_size_initial / batch_size_div)
train_size = int(int(train_size_initial / batch_size) * batch_size) # provide even division of training / batches
val_size = int(int((data_size - train_size) / batch_size) * batch_size) # provide even division of val / batches
print('Data Size: {} Train Size: {} Batch Size: {}'.format(data_size, train_size, batch_size))
Split data into train and validation sets
train, val = x_samples[0:train_size, 0:-1], x_samples[train_size:train_size + val_size, 0:-1]

Answer 4

Both training and validation data need to be divisible by the batch size. Make sure that any part of the model using batch size takes the same number (e.g. batch_input_shape in your LSTM layer, and batch_size in model.fit() and model.predict(). Down-sample training and validation data if need be to make this work.
e.g.
>>> batch_size = 100
>>> print(x_samples_train.shape)
>>> print(x_samples_validation.shape)
(42028, 24, 14)
(10451, 24, 14)
# Down-sample so training and validation are both divisible by batch_size
>>> x_samples_train_ds = x_samples_train[-42000:]
>>> print(x_samples_train_ds.shape)
>>> y_samples_train_ds = y_samples_train[-42000:]
>>> print(y_samples_train_ds.shape)
(42000, 24, 14)
(42000,)
>>> x_samples_validation_ds = x_samples_validation[-10000:]
>>> print(x_samples_validation_ds.shape)
>>> y_samples_validation_ds = y_samples_validation[-10000:]
>>> print(y_samples_validation_ds.shape)
(10000, 24, 14)
(10000,)