I'm creating an LSTM that predicts next words for text generation, but I keep getting an error when trying to train the model
I've tried changing the dimensions of the input but that doesn't seem to work
X_train.shape
(32249, 5, 1)
y_train.shape
(32249,)
print(X_train[0])
[['opened'] ['for'] ['it.'] ['how'] ['they']]
model = Sequential()
model.add(Bidirectional(LSTM(128), input_shape=(5, 1)))
model.add(Dropout(0.2))
model.add(Dense(units = len(words)))
model.add(Activation('softmax'))
model.compile(optimizer="adam", loss="mean_squared_error")
model.fit(X_train, y_train, epochs=1)
ValueError: Error when checking target: expected activation_27 to have shape (2943,) but got array with shape (1,)
I've shown the code, and the results it gives. I keep getting this error ^
Related
I am trying to setup an LSTM in order to feed it with my numpy array features and labels.
Here is my first attempt:
nb_features =len(seq_cols)
print("initial shape:", X_train.shape)
print("nb features", nb_features)
# X_train = X_train.reshape(X_train.shape + (1,))
print("Seq length ", seq_length)
print('New shape ', X_train.shape)
model = Sequential()
model.add(LSTM(
input_shape=(nb_features, 1),
units=100,
return_sequences=True))
model.add(Dropout(0.2))
model.add(Dense(units=1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X_train, y_train, epochs=10, batch_size=200, verbose=1,
callbacks = [EarlyStopping(monitor='val_loss', min_delta=0, patience=0, verbose=0, mode='auto')])
Which gives me the output
initial shape: (175850, 4)
nb features 4
Seq length 50
New shape (175850, 4)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-99-50959413cb62> in <module>()
1 model.fit(X_train, y_train, epochs=10, batch_size=200, verbose=1,
----> 2 callbacks = [EarlyStopping(monitor='val_loss', min_delta=0, patience=0, verbose=0, mode='auto')])
2 frames
/usr/local/lib/python3.6/dist-packages/keras/engine/training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
126 ': expected ' + names[i] + ' to have ' +
127 str(len(shape)) + ' dimensions, but got array '
--> 128 'with shape ' + str(data_shape))
129 if not check_batch_axis:
130 data_shape = data_shape[1:]
ValueError: Error when checking input: expected lstm_28_input to have 3 dimensions, but got array with shape (175850, 4)
So I am trying to reshape by uncommenting line 4
nb_features =len(seq_cols)
print("initial shape:", X_train.shape)
print("nb features", nb_features)
X_train = X_train.reshape(X_train.shape + (1,))
print("Seq length ", seq_length)
print('New shape ', X_train.shape)
model = Sequential()
model.add(LSTM(
input_shape=(nb_features, 1),
units=100,
return_sequences=True))
model.add(Dropout(0.2))
model.add(Dense(units=1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X_train, y_train, epochs=10, batch_size=200, verbose=1,
callbacks = [EarlyStopping(monitor='val_loss', min_delta=0, patience=0, verbose=0, mode='auto')])
which now gives me the error that I have wrong dimensions.
initial shape: (175850, 4)
nb features 4
Seq length 50
New shape (175850, 4, 1)
ValueError: Error when checking input: expected lstm_28_input to have 3 dimensions, but got array with shape (175850, 4)
I think I am just wondering around doing random changes.
Can anyone please give me an insight on what pieces I am missing from the puzzle? I am a beginner in the field and the errors are not helping me much.
P.S: X_train is a numpy array
Keras input_shape ignores the first dimension because it indicates the number of training examples, m. This is because Keras is able to work with any number of training examples, it only cares about the actual input dimensions.
For example, input_shape=(nb_features, 1)=(4,1) means it is expecting the input to be (None, 4, 1), where none is the number of training examples. You can also see this by typing model.summary() after you compile, but before you fit.
This is 3 dimensions, hence the "expected lstm_28_input to have 3 dimensions" error. You're feeding it (175850, 4) which is a two dimensional array.
I'm working on a speech recognition problem running on Colab using LSTM. The audio files were converted into spectrograms and then normalized. There are 6840 spectrograms in total and the shape of each one is (288, 864, 4).
I already tried a few examples with RNN and CNN and they worked, but when I try an example using a LSTM I get shape errors, every time either there is one more or one less dimension than expected. Here are some of these cases :
rnn = keras.Sequential()
rnn.add(keras.layers.SimpleRNN(500, input_shape = (864, 4)))
rnn.add(keras.layers.LSTM(500, return_sequences = True))
rnn.add(keras.layers.Dropout(0.2))
rnn.add(keras.layers.LSTM(500, return_sequences = True))
rnn.add(keras.layers.Dropout(0.2))
rnn.add(keras.layers.LSTM(500, return_sequences = True))
rnn.add(keras.layers.Dropout(0.2))
rnn.add(keras.layers.Dense(212, activation = 'softmax'))
rnn.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy',metrics = ['accuracy'])
rnn.fit(X_train, y_train, epochs = 5, validation_data=(X_test, y_test))
scores = rnn.evaluate(X_test, y_test, verbose=1)
print('Test loss:', scores[0])
print('Test accuracy:', '%.2f' % (scores[1] * 100), '%')
The following error is raised on the first LSTM layer : ValueError: Input 0 of layer lstm_54 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [None, 500]
If I remove the SimpleRNN line and feed the input directly to the first LSTM like this
rnn.add(keras.layers.LSTM(500, return_sequences = True, input_shape = (288, 864, 4)))
I get : ValueError: Input 0 of layer lstm_56 is incompatible with the layer: expected ndim=3, found ndim=4. Full shape received: [None, 288, 864, 4]
I tried reshaping the images to (4, 288 * 864) and got the same error when trying to use the RNN layer, but with just the LSTM I got InvalidArgumentError: Incompatible shapes: [32] vs. [32,4].
No idea where the 32 came from, though.
One last thing, not really an issue but more of a request, is there any library that can resize images the simple way? 288x864 is too big for Colab, so I'll have to do it eventually to be able to load all 6840 images and feed it to the neural network. Right now I'm just using 100 samples to test.
Feel free to leave suggestions about other methods, cabalistic number of nodes/layers or anything like that.
LSTM input is 3 dimensions [n_samples, n_timesteps, n_features], so your first line also need to enable return sequences:
rnn.add(keras.layers.SimpleRNN(500, return_sequences = True, input_shape = (864, 4)))
Next, your Dense layer will complain from wrong input size, so you want to remove return_sequence on the last LSTM network:
rnn.add(keras.layers.LSTM(500))
If you still want to keep the return_sequences = True on the last LSTM layer, you might want to wrap the Dense layer in a TimeDistributed.
I tried it on the following input and they seems to work
X_train = np.random.rand(100, 864, 4)
y_train = np.random.rand(100, 1)
The PIL from pillow package has plenty of image manipulation methods.
My x_train shape is (798,3) and y_train input shape is (798, 1). I am creating a RNN like this
def create_rnn_model():
model = Sequential()
model.add(SimpleRNN(20,return_sequences=False,stateful=stateful,activation='relu',batch_input_shape=(1,3,1)))
model.add(Activation('relu'))
adam = optimizers.Adam(lr=0.001)
model.compile(loss='mean_squared_error', optimizer=adam, metrics=[root_mean_squared_error])
return model
But this returns the error
ValueError: Error when checking input: expected simple_rnn_1_input to
have 3 dimensions, but got array with shape (798, 3)
My batch size =1 and my timestep is 3 and dat_dim=1 .Then where am I doing it wrong?
Any help is appreciated.
EDIT
I changed my x_train to shape (798,3,1) and y_train shape to (798,) and ran the model but it threw an error
ValueError: Error when checking target: expected activation_1 to have shape (20,) but got array with shape (1,)
But I can run the model with 1 unit. How do I specify the model to run with 20 units instead of one.
My Network Model is like this and my current L=3.
I am trying to find out label associated with word from annotated text. I am using a bidirectional LSTM. I have X_train which is having shape (1676, 39) and Y_train with the same shape (1676, 39).
input = Input(shape=(sequence_length,))
model = Embedding(input_dim=n_words, output_dim=20,
input_length=sequence_length, mask_zero=True)(input)
model = Bidirectional(LSTM(units=50, return_sequences=True,
recurrent_dropout=0.1))(model)
out_model = TimeDistributed(Dense(50, activation="softmax"))(model)
model = Model(input, out_model)
model.compile(optimizer="rmsprop", loss= "categorical_crossentropy", metrics=["accuracy"])
model.fit(X_train, Y_train, batch_size=32, epochs= 10,
validation_split=0.1)
While executing this, I am getting error:
ValueError: Error when checking target: expected time_distributed_5 to have 3 dimensions, but got array with shape (1676, 39).
I am not able to find out how to feed proper dimension which is needed by the Keras LSTM model.
In the LSTM you set return_sequences=True, as a result, the outputs of the layer is a Tensor with shape of [batch_size * 39 * 50]. Then you pass this Tensor to TimeDistributed layer. TimeDistributed apply Dense layer on the each time stamp. The outputs of the layer, again is [batch_size * 39 * 50]. As you see, you pass 3 dimension Tensor for prediction, while your ground truth is 2 dimension (1676, 39).
How to fix the issue?
1) Remove return_sequences=True from LSTM args.
2) Remove TimeDistributed layer and apply Dense layer directly.
inps = keras.layers.Input(shape=(39,))
embedding = keras.layers.Embedding(vocab_size, 16)(inps)
rnn = keras.layers.LSTM(50)(embedding)
dense = keras.layers.Dense(50, activation="softmax")(rnn)
prediction = keras.layers.Dense(39, activation='softmax')(dense)
Model :
model = Sequential()
act = 'relu'
model.add(Dense(430, input_shape=(3,)))
model.add(Activation(act))
model.add(Dense(256))
model.add(Activation(act))
model.add(Dropout(0.4))
model.add(Dense(148))
model.add(Activation(act))
model.add(Dropout(0.3))
model.add(Dense(1))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
#model.summary()
Error :
Error when checking target: expected activation_4 to have shape (None, 1) but got array with shape (1715, 2)
The error is in the last layer of the neural network. The neural network is trying to classify whether 2 drugs are synergic or not.
COMPLETE SOURCE CODE :
https://github.com/tanmay-edgelord/Drug-Synergy-Models/blob/master/Drug%20Synergy%20NN%20Classifier.ipynb
Data:
https://github.com/tanmay-edgelord/Drug-Synergy-Models/blob/master/train.csv
In your code you have the following lines:
y_binary = to_categorical(Y[:])
y_train = y_binary[:split]
y_test = y_binary[split:]
to_categorical transforms the vector into a 1-hot vector. So since you have two classes, it transforms every number to a vector of length 2 (0 is transformed to [1,0] and 1 is transformed to [0,1]). So your last layer needs to be defined as follows:
model.add(Dense(2))
model.add(Activation('softmax'))
(note that I replaced the 1 with 2).