My x_train shape is (798,3) and y_train input shape is (798, 1). I am creating a RNN like this
def create_rnn_model():
model = Sequential()
model.add(SimpleRNN(20,return_sequences=False,stateful=stateful,activation='relu',batch_input_shape=(1,3,1)))
model.add(Activation('relu'))
adam = optimizers.Adam(lr=0.001)
model.compile(loss='mean_squared_error', optimizer=adam, metrics=[root_mean_squared_error])
return model
But this returns the error
ValueError: Error when checking input: expected simple_rnn_1_input to
have 3 dimensions, but got array with shape (798, 3)
My batch size =1 and my timestep is 3 and dat_dim=1 .Then where am I doing it wrong?
Any help is appreciated.
EDIT
I changed my x_train to shape (798,3,1) and y_train shape to (798,) and ran the model but it threw an error
ValueError: Error when checking target: expected activation_1 to have shape (20,) but got array with shape (1,)
But I can run the model with 1 unit. How do I specify the model to run with 20 units instead of one.
My Network Model is like this and my current L=3.
Related
I'm trying to develop as classifier for two classes. I've implemented the model as follows:
model = keras.models.Sequential() #using tensorflow's version of keras
model.add(keras.layers.InputLayer(input_shape = X_train_scaled[:,1].shape))
model.add(keras.layers.Dense(250,activation="relu"))
model.add(keras.layers.Dense(50,activation="relu"))
model.add(keras.layers.Dense(2,activation="softmax"))
model.summary()
# Compile the model
model.compile(loss = 'sparse_categorical_crossentropy',
optimizer = "sgd",
metrics = ["accuracy"])
The size of the inputs are
X_train_scaled[:,1].shape, y_train.shape
((552,), (552,))
The entire error message is:
ValueError: Input 0 of layer sequential_9 is incompatible with the layer:
expected axis -1 of input shape to have value 552 but received input with shape (None, 1)
What am I doing wrong here?
The error message says that you defined a model which expects as input a shape of (batch_size, 552) and you are trying to feed it an array with a shape of (batch_size, 1).
The issue is most likely with
input_shape = X_train_scaled[:,1].shape)
This should most likely be:
input_shape = X_train_scaled.shape[1:]
i.e. you want to define the shape of your model to the the shape of the features (without the number of examples). The model is then fed in mini-batches... e.g. if you do call model.fit(X_train_scaled, ...) keras will create mini-batches (of 32 examples by default) and update the model weights for each mini batch.
Also, please be aware that you defined the model to return a shape of (batch_size, 2). So y_train must have a shape of (X_train.shape[0], 2).
The question was answered by Pedro
I need outputs at every recurrent layer and my setup is as follows:
100 training examples, 3 time steps per example, and 20-d feature vector for each individual element.
x_train: (100,3,20)
y_train: (100,20)
LSTM architecture:
model.add(LSTM(20, input_shape=(3,20), return_sequences=True))
model.compile(loss='mean_absolute_error', optimizer='adam', metrics=['accuracy'])
model.summary()
Training:
history = model.fit(x_train, y_train, epochs=50, validation_data=(x_test, y_test))
Error:
ValueError: Dimensions must be equal, but are 20 and 3 for '{{node Equal}} = Equal[T=DT_FLOAT, incompatible_shape_error=true](IteratorGetNext:1, Cast_1)' with input shapes: [?,20], [?,3].
Please help me with the correct input/output LSTM dimensions.
Thanks
LSTM(20, input_shape=(3,20), return_sequences=True) takes as input shape (100,3,20) and returns (100,3,20). Your target output is however encoded as (100,20).
From the dimensions, I assume you want to map each sequence to a non-sequence, i.e. you can do:
LSTM(20, input_shape=(3,20), return_sequences=False)
This will return the final hidden state, i.e. a shape of (100,20) which matches your target output.
I'm trying out different Neural Network architectures for a word based NLP.
So far I've used bidirectional-, embedded- and models with GRU's guided by this tutorial: https://towardsdatascience.com/language-translation-with-rnns-d84d43b40571 and it all worked out well.
When I tried using LSTM's however, I get an error saying:
logits and labels must have the same first dimension, got logits shape [32,186] and labels shape [4704]
How can I solve this?
My source and target dataset consists of 7200 sample sentences. They are integer tokenized and embedded. The source dataset is post padded to match the length of the target dataset.
Here is my model and the relevant code:
lstm_model = Sequential()
lstm_model.add(Embedding(src_vocab_size, 128, input_length=X.shape[1], input_shape=X.shape[1:]))
lstm_model.add(LSTM(128, return_sequences=False, dropout=0.1, recurrent_dropout=0.1))
lstm_model.add(Dense(128, activation='relu'))
lstm_model.add(Dropout(0.5))
lstm_model.add((Dense(target_vocab_size, activation='softmax')))
lstm_model.compile(optimizer=Adam(0.002), loss='sparse_categorical_crossentropy', metrics=['accuracy'])
history = lstm_model.fit(X, Y, batch_size = 32, callbacks=CALLBACK, epochs = 100, validation_split = 0.25) #At this line the error is raised!
With the shapes:
X.shape = (7200, 147)
Y.shape = (7200, 147, 1)
src_vocab_size = 188
target_vocab_size = 186
I've looked at similar question on here already and tried adding a Reshape layer
simple_lstm_model.add(Reshape((-1,)))
but this only causes the following error:
"TypeError: __int__ returned non-int (type NoneType)"
It's really weird as I preprocess the dataset the same way for all models and it works just fine except for the above.
You should have return_sequences=True and return_state=False in calling the LSTM constructor.
In your snippet, the LSTM only return its last state, instead of the sequence of states for every input embedding. In theory, you could have spotted it from the error message:
logits and labels must have the same first dimension, got logits shape [32,186] and labels shape [4704]
The logits should be three-dimensional: batch size × sequence length × number of classes. The length of the sequences is 147 and indeed 32 × 147 = 4704 (number of your labels). This could have told you the length of the sequences disappeared.
I'm creating an LSTM that predicts next words for text generation, but I keep getting an error when trying to train the model
I've tried changing the dimensions of the input but that doesn't seem to work
X_train.shape
(32249, 5, 1)
y_train.shape
(32249,)
print(X_train[0])
[['opened'] ['for'] ['it.'] ['how'] ['they']]
model = Sequential()
model.add(Bidirectional(LSTM(128), input_shape=(5, 1)))
model.add(Dropout(0.2))
model.add(Dense(units = len(words)))
model.add(Activation('softmax'))
model.compile(optimizer="adam", loss="mean_squared_error")
model.fit(X_train, y_train, epochs=1)
ValueError: Error when checking target: expected activation_27 to have shape (2943,) but got array with shape (1,)
I've shown the code, and the results it gives. I keep getting this error ^
I am trying to find out label associated with word from annotated text. I am using a bidirectional LSTM. I have X_train which is having shape (1676, 39) and Y_train with the same shape (1676, 39).
input = Input(shape=(sequence_length,))
model = Embedding(input_dim=n_words, output_dim=20,
input_length=sequence_length, mask_zero=True)(input)
model = Bidirectional(LSTM(units=50, return_sequences=True,
recurrent_dropout=0.1))(model)
out_model = TimeDistributed(Dense(50, activation="softmax"))(model)
model = Model(input, out_model)
model.compile(optimizer="rmsprop", loss= "categorical_crossentropy", metrics=["accuracy"])
model.fit(X_train, Y_train, batch_size=32, epochs= 10,
validation_split=0.1)
While executing this, I am getting error:
ValueError: Error when checking target: expected time_distributed_5 to have 3 dimensions, but got array with shape (1676, 39).
I am not able to find out how to feed proper dimension which is needed by the Keras LSTM model.
In the LSTM you set return_sequences=True, as a result, the outputs of the layer is a Tensor with shape of [batch_size * 39 * 50]. Then you pass this Tensor to TimeDistributed layer. TimeDistributed apply Dense layer on the each time stamp. The outputs of the layer, again is [batch_size * 39 * 50]. As you see, you pass 3 dimension Tensor for prediction, while your ground truth is 2 dimension (1676, 39).
How to fix the issue?
1) Remove return_sequences=True from LSTM args.
2) Remove TimeDistributed layer and apply Dense layer directly.
inps = keras.layers.Input(shape=(39,))
embedding = keras.layers.Embedding(vocab_size, 16)(inps)
rnn = keras.layers.LSTM(50)(embedding)
dense = keras.layers.Dense(50, activation="softmax")(rnn)
prediction = keras.layers.Dense(39, activation='softmax')(dense)