Trying to add validation to model.fit, but whenever I do I get an error:
ValueError: Cannot feed value of shape (6, 4, 10) for Tensor 'lstm_input_1:0', which has shape '(32, 4, 10)'
Model:
data_dim = 10
timesteps = 4
batch_size = 32
model = Sequential()
model.add(LSTM(128, batch_input_shape=(batch_size, timesteps, data_dim), return_sequences=True, stateful=True))
model.add(LSTM(64, return_sequences=True, stateful=True))
model.add(LSTM(32, stateful=True))
model.add(Dense(2, activation='softmax'))
sgd = SGD(lr=0.001, momentum=0.0, decay=0.0, nesterov=False)
model.compile(optimizer=sgd, loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(x_train, y_train, nb_epoch=50, batch_size=batch_size, validation_split=0.5)
What could be the error? If I remove validation_split the training works just fine. I've also tried to manually split my training set into two and add it with validation_data=(x_val, y_val) but I got the exact same error.
The issue comes from the fact that you hard code the batch_size value of your inputs. You have fixed it to 32 and then when you try and validate your model, the validation data is sent with a batch of 6 samples, this might be because you don't have enough validation data or maybe because the number of sample isn't a multiple of 32... However, I would let the batch_size free if I was you. Like this:
model.add(LSTM(128, input_shape=(timesteps, data_dim), return_sequences=True, stateful=True))
You specify input_shape instead of batch_input_shape. That way, your network will accept any size of batch, every layer down in the stream of your model are made to adapt to any batch_size if not hardcoded.
I hope this helps :)
Related
I've got myself into a bit of trouble. I've got 4 features, and I want to predict each one of them at the same time. My lookback is 12 and I want to predict 12 timesteps ahead. Is it possible to predict all the 4 targets in parallel?
I have to following piece of code. Shape on train_df is (40000, 4) and val_df is (8000, 4).
win_length=12
batch=32
n_features=4
train_generator = TimeseriesGenerator(train_df, train_df, length=win_length, sampling_rate=1, batch_size=batch)
val_generator = TimeseriesGenerator(val_df, val_df, length=win_length, sampling_rate=1, batch_size=batch)
model = Sequential()
model.add(LSTM(128, activation='tanh', input_shape=(win_length, n_features), return_sequences=True))
model.add(LSTM(128, activation='tanh', return_sequences=True))
model.add(LSTM(64, activation='tanh', return_sequences=True))
model.add(TimeDistributed(Dense(1)))
model.compile(loss='mse', optimizer='adam')
model.summary()
model.fit_generator(train_generator, validation_data=val_generator)
I get the following error from the fit_generator-function, and I can't seem to figure out how so. Any ideas?
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [32,12] vs. [32,4]
I don't quite understand how TimeseriesGenerator works, but it seems like you want to have two same sequences for data and target, hence the mse. I've checked myself and TimeseriesGenerator doesn't produce two same sequences (data with shape (32, 12, 4) and targets with shape (32, 4)). My best bet right now is to implement the generator manually. Also I think model.add(TimeDistributed(Dense(1))) should be changed to model.add(TimeDistributed(Dense(4)))
I have trained a keras LSTM model. But after training, all i get is the final parameters of the models after training with 10 epochs and batch size=120. How can i get intermediate parameter after a batch keras.
Example: after 120 sample in each batch i can get the intermediate parameter of this step.
I have tried callback method and backend in keras, but i do not know how to get the
'''python
model = Sequential()
model.add(Embedding(max_features, 32))
#model.add(LSTM(32, return_sequences=True, input_shape=(1,texts.shape[0])))
model.add(LSTM(32))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['acc'])
history_ltsm = model.fit(texts_train, y_train, epochs=10, batch_size=120, validation_split=0.2)
'''
I expected the model run step by step based on each batch to show the intermediate parameters, but not the all epochs.
Thank you very much!
I trained a neural network with 37 Inputs. It has around 85% accuracy. Is it possible for me to find out which Input has the most effect. I tried this code but I cannot figure out how to find most important Input
weights = model.layers[0].get_weights()[0]
biases = model.layers[0].get_weights()[1]
One possible solution is to wrap your model with keras.wrappers.scikit_learn and then use Recursive Feature elimination in scikit-learn:
def create_model():
# create model
model = Sequential()
model.add(Dense(512, activation='relu'))
model.add(Dense(512, activation='relu'))
model.add(Dense(10, activation='softmax'))
# Compile model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
model = KerasClassifier(build_fn=create_model, epochs=100, batch_size=128, verbose=0)
rfe = RFE(estimator=model, n_features_to_select=1, step=1)
rfe.fit(X, y)
ranking = rfe.ranking_.reshape(digits.images[0].shape)
# Plot pixel ranking
plt.matshow(ranking, cmap=plt.cm.Blues)
plt.colorbar()
plt.title("Ranking of pixels with RFE")
plt.show()
If you need to visualize weights see here.
I don't understand input of RNN model. Why it show None before node size every layer. Why it is (None,1) (None,12)
This is my code.
K.clear_session()
model = Sequential()
model.add(Dense(12, input_dim=1, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
model.summary()
This is not a RNN, it's just a fully connected network (FC or Dense).
The first dimension of every tensor in a Keras network is the batch_size, which represents the number of "samples" or "examples" you are passing to the model. The value is None because this dimension is not fixed, you can have batches of any size you want.
I am building a RNN to predict a many to one question.
#Input_X:
[
[1,2,3,4,5,6,7,8,9,10],
[2,3,4,5,6,7,8,9,10,11]
]
#Input_Y:
[
11,
12
]
#Each number represent a category
X=np.reshape(Input_X,(len(Input_X), 10, 1))
y=np.utils.to_catgeorical(Input_Y) #one hot encode,
My model setup:
#####This works
model=Sequential()
model.add(LSTM(256, input_shape(X.shape[1], X.shape[2])))
model.add(Dense(y.shape[1], activation='softmax'))
model.compile(loss='categorical_crossentrophy', optimizer='adam', metrics=['accuracy'])
model.fit(X,y, ........)
I want to try the TimeDistributed(Dense) layer instead, for example: https://keras.io/layers/wrappers/. So I changed above to below:
model=Sequential()
model.add(LSTM(256, input_shape(X.shape[1], X.shape[2])))
model.add(TimeDistributed(Dense(y.shape[1], activation='softmax')))
model.compile(loss='categorical_crossentrophy', optimizer='adam', metrics=['accuracy'])
model.fit(X,y, ........)
I am getting a AssertionError. Which report the matrix size is not what expected.
What steps I missed?
I think you need to add return_sequences=True to the LSTM cell
```
model=Sequential()
model.add(LSTM(256, return_sequences=True, input_shape(X.shape[1], X.shape[2])))
model.add(TimeDistributed(Dense(y.shape[1], activation='softmax')))
model.compile(loss='categorical_crossentrophy', optimizer='adam', metrics=['accuracy'])
model.fit(X,y, ........)
```
return_sequences=True works for me.
In the OP's question, y.shape is (2,1) which has 2 samples and only 1 feature , so it's not suited for Many to Many model.