I have been trying to stack Convolutional neural networks with GRUs for an image to text problem.
Here's my model :
model=Sequential()
model.add(TimeDistributed(Conv2D(16,kernel_size
(3,3),data_format="channels_last",input_shape=
(129,80,564,3),padding='SAME',strides=(1,1))))
model.add(TimeDistributed(Activation("relu")))
model.add(TimeDistributed(Conv2D(16,kernel_size =(3,3),strides=(1,1))))
model.add(TimeDistributed(Activation("relu")))
model.add(TimeDistributed(MaxPooling2D(pool_size=2,strides=(1,1) )))
model.add(TimeDistributed(Reshape((280*38*16,))))
model.add(TimeDistributed(Dense(32)))
model.add(GRU(512))
model.add(Dense(50))
model.add(Activation("softmax"))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=
['accuracy'])
When I try to fit my model I get the following error :
-------------------------------------------------------------------------
ValueError Traceback (most recent call
last)
<ipython-input-125-c6a3c418689c> in <module>()
1 nb_epoch = 100
----> 2 model.fit(X2,L2, epochs=100)
10 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/nn_ops.py
in _get_sequence(value, n, channel_index, name)
71 else:
72 raise ValueError("{} should be of length 1, {} or {} but was
{}".format(
---> 73 name, n, n + 2, current_n))
74
75 if channel_index == 1:
ValueError: strides should be of length 1, 1 or 3 but was 2
I cannot even begin to wrap my head around why this message appears.I have specified the "strides" parameters for all layers. Any help will be deeply appreciated.
P.S - I did not have any problems when I tried to fit a model without TimeDistributed layers. Maybe there is something to do with this that raises this error.
You have made several mistakes in your code.
In the first layer you should specify the input_shape of the TimeDistributed layer, not Conv2D layer.
MaxPooling2D is used for down-sampling the images spatial size. but with strides=(1,1) the image size will remain same and not be reduced.
Using padding='SAME' in the first layer will add zero-padding while doing convolution and will result in a shape mismatch error in the Reshape layer. Instead you can use Flatten layer.
Default value of stride in a Conv2D is strides=(1,1), so, it's optional to mention.
Finally, the working code should be something following:
model=keras.models.Sequential()
model.add(keras.layers.TimeDistributed(keras.layers.Conv2D(16, kernel_size=(3,3), data_format="channels_last"),input_shape=(129,80,564,3)))
model.add(keras.layers.TimeDistributed(keras.layers.Activation("relu")))
model.add(keras.layers.TimeDistributed(keras.layers.Conv2D(16, kernel_size =(3,3))))
model.add(keras.layers.TimeDistributed(keras.layers.Activation("relu")))
model.add(keras.layers.TimeDistributed(keras.layers.MaxPooling2D(pool_size=2)))
# model.add(keras.layers.TimeDistributed(keras.layers.Flatten()))
model.add(keras.layers.TimeDistributed(keras.layers.Reshape((280*38*16,))))
model.add(keras.layers.TimeDistributed(keras.layers.Dense(32)))
model.add(keras.layers.GRU(512))
model.add(keras.layers.Dense(50))
model.add(keras.layers.Activation("softmax"))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics= ['accuracy'])
Related
I'm trying out different Neural Network architectures for a word based NLP.
So far I've used bidirectional-, embedded- and models with GRU's guided by this tutorial: https://towardsdatascience.com/language-translation-with-rnns-d84d43b40571 and it all worked out well.
When I tried using LSTM's however, I get an error saying:
logits and labels must have the same first dimension, got logits shape [32,186] and labels shape [4704]
How can I solve this?
My source and target dataset consists of 7200 sample sentences. They are integer tokenized and embedded. The source dataset is post padded to match the length of the target dataset.
Here is my model and the relevant code:
lstm_model = Sequential()
lstm_model.add(Embedding(src_vocab_size, 128, input_length=X.shape[1], input_shape=X.shape[1:]))
lstm_model.add(LSTM(128, return_sequences=False, dropout=0.1, recurrent_dropout=0.1))
lstm_model.add(Dense(128, activation='relu'))
lstm_model.add(Dropout(0.5))
lstm_model.add((Dense(target_vocab_size, activation='softmax')))
lstm_model.compile(optimizer=Adam(0.002), loss='sparse_categorical_crossentropy', metrics=['accuracy'])
history = lstm_model.fit(X, Y, batch_size = 32, callbacks=CALLBACK, epochs = 100, validation_split = 0.25) #At this line the error is raised!
With the shapes:
X.shape = (7200, 147)
Y.shape = (7200, 147, 1)
src_vocab_size = 188
target_vocab_size = 186
I've looked at similar question on here already and tried adding a Reshape layer
simple_lstm_model.add(Reshape((-1,)))
but this only causes the following error:
"TypeError: __int__ returned non-int (type NoneType)"
It's really weird as I preprocess the dataset the same way for all models and it works just fine except for the above.
You should have return_sequences=True and return_state=False in calling the LSTM constructor.
In your snippet, the LSTM only return its last state, instead of the sequence of states for every input embedding. In theory, you could have spotted it from the error message:
logits and labels must have the same first dimension, got logits shape [32,186] and labels shape [4704]
The logits should be three-dimensional: batch size × sequence length × number of classes. The length of the sequences is 147 and indeed 32 × 147 = 4704 (number of your labels). This could have told you the length of the sequences disappeared.
I am trying to setup an LSTM in order to feed it with my numpy array features and labels.
Here is my first attempt:
nb_features =len(seq_cols)
print("initial shape:", X_train.shape)
print("nb features", nb_features)
# X_train = X_train.reshape(X_train.shape + (1,))
print("Seq length ", seq_length)
print('New shape ', X_train.shape)
model = Sequential()
model.add(LSTM(
input_shape=(nb_features, 1),
units=100,
return_sequences=True))
model.add(Dropout(0.2))
model.add(Dense(units=1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X_train, y_train, epochs=10, batch_size=200, verbose=1,
callbacks = [EarlyStopping(monitor='val_loss', min_delta=0, patience=0, verbose=0, mode='auto')])
Which gives me the output
initial shape: (175850, 4)
nb features 4
Seq length 50
New shape (175850, 4)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-99-50959413cb62> in <module>()
1 model.fit(X_train, y_train, epochs=10, batch_size=200, verbose=1,
----> 2 callbacks = [EarlyStopping(monitor='val_loss', min_delta=0, patience=0, verbose=0, mode='auto')])
2 frames
/usr/local/lib/python3.6/dist-packages/keras/engine/training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
126 ': expected ' + names[i] + ' to have ' +
127 str(len(shape)) + ' dimensions, but got array '
--> 128 'with shape ' + str(data_shape))
129 if not check_batch_axis:
130 data_shape = data_shape[1:]
ValueError: Error when checking input: expected lstm_28_input to have 3 dimensions, but got array with shape (175850, 4)
So I am trying to reshape by uncommenting line 4
nb_features =len(seq_cols)
print("initial shape:", X_train.shape)
print("nb features", nb_features)
X_train = X_train.reshape(X_train.shape + (1,))
print("Seq length ", seq_length)
print('New shape ', X_train.shape)
model = Sequential()
model.add(LSTM(
input_shape=(nb_features, 1),
units=100,
return_sequences=True))
model.add(Dropout(0.2))
model.add(Dense(units=1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X_train, y_train, epochs=10, batch_size=200, verbose=1,
callbacks = [EarlyStopping(monitor='val_loss', min_delta=0, patience=0, verbose=0, mode='auto')])
which now gives me the error that I have wrong dimensions.
initial shape: (175850, 4)
nb features 4
Seq length 50
New shape (175850, 4, 1)
ValueError: Error when checking input: expected lstm_28_input to have 3 dimensions, but got array with shape (175850, 4)
I think I am just wondering around doing random changes.
Can anyone please give me an insight on what pieces I am missing from the puzzle? I am a beginner in the field and the errors are not helping me much.
P.S: X_train is a numpy array
Keras input_shape ignores the first dimension because it indicates the number of training examples, m. This is because Keras is able to work with any number of training examples, it only cares about the actual input dimensions.
For example, input_shape=(nb_features, 1)=(4,1) means it is expecting the input to be (None, 4, 1), where none is the number of training examples. You can also see this by typing model.summary() after you compile, but before you fit.
This is 3 dimensions, hence the "expected lstm_28_input to have 3 dimensions" error. You're feeding it (175850, 4) which is a two dimensional array.
I'm working on a speech recognition problem running on Colab using LSTM. The audio files were converted into spectrograms and then normalized. There are 6840 spectrograms in total and the shape of each one is (288, 864, 4).
I already tried a few examples with RNN and CNN and they worked, but when I try an example using a LSTM I get shape errors, every time either there is one more or one less dimension than expected. Here are some of these cases :
rnn = keras.Sequential()
rnn.add(keras.layers.SimpleRNN(500, input_shape = (864, 4)))
rnn.add(keras.layers.LSTM(500, return_sequences = True))
rnn.add(keras.layers.Dropout(0.2))
rnn.add(keras.layers.LSTM(500, return_sequences = True))
rnn.add(keras.layers.Dropout(0.2))
rnn.add(keras.layers.LSTM(500, return_sequences = True))
rnn.add(keras.layers.Dropout(0.2))
rnn.add(keras.layers.Dense(212, activation = 'softmax'))
rnn.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy',metrics = ['accuracy'])
rnn.fit(X_train, y_train, epochs = 5, validation_data=(X_test, y_test))
scores = rnn.evaluate(X_test, y_test, verbose=1)
print('Test loss:', scores[0])
print('Test accuracy:', '%.2f' % (scores[1] * 100), '%')
The following error is raised on the first LSTM layer : ValueError: Input 0 of layer lstm_54 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [None, 500]
If I remove the SimpleRNN line and feed the input directly to the first LSTM like this
rnn.add(keras.layers.LSTM(500, return_sequences = True, input_shape = (288, 864, 4)))
I get : ValueError: Input 0 of layer lstm_56 is incompatible with the layer: expected ndim=3, found ndim=4. Full shape received: [None, 288, 864, 4]
I tried reshaping the images to (4, 288 * 864) and got the same error when trying to use the RNN layer, but with just the LSTM I got InvalidArgumentError: Incompatible shapes: [32] vs. [32,4].
No idea where the 32 came from, though.
One last thing, not really an issue but more of a request, is there any library that can resize images the simple way? 288x864 is too big for Colab, so I'll have to do it eventually to be able to load all 6840 images and feed it to the neural network. Right now I'm just using 100 samples to test.
Feel free to leave suggestions about other methods, cabalistic number of nodes/layers or anything like that.
LSTM input is 3 dimensions [n_samples, n_timesteps, n_features], so your first line also need to enable return sequences:
rnn.add(keras.layers.SimpleRNN(500, return_sequences = True, input_shape = (864, 4)))
Next, your Dense layer will complain from wrong input size, so you want to remove return_sequence on the last LSTM network:
rnn.add(keras.layers.LSTM(500))
If you still want to keep the return_sequences = True on the last LSTM layer, you might want to wrap the Dense layer in a TimeDistributed.
I tried it on the following input and they seems to work
X_train = np.random.rand(100, 864, 4)
y_train = np.random.rand(100, 1)
The PIL from pillow package has plenty of image manipulation methods.
This error is generated when I try to run the training step. The dataset is MNIST dataset from Kaggle. I'm using a neural network to predict the handwritten digits:
Input Data : [33600, 784] reshaped into [784, 33600]
Neural network architecture:
Layer 1 has W1 1000 by 784 relu
Layer 2 has W2 1000 by 1000 relu
Layer 3 has W3 500 by 1000 relu
Layer 4 has W4 200 by 500 relu
Layer 5 has W5 10 by 200 with softmax
No biases used
Code:
print(X_train[:, 0].reshape(-1, 1).shape," ",y_train[:,0].reshape(-1,1).shape)`
Output: (784, 1) (10, 1)
Code:
X, Y = tf.placeholder(tf.float32,[784, None]), tf.placeholder(tf.float32,[10, None])
logits = forward_propagation(X, parameters)
cost = compute_cost(logits, Y)
optimizer = tf.train.AdamOptimizer(learning_rate=1e-3).minimize(cost)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
_,c = sess.run([optimizer,cost], feed_dict= {X:X_train[:,0].reshape(-1,1),
Y:y_train[:, 0].reshape(-1,1)})
print(c)
Output:
ValueError Traceback (most recent call
last)
<ipython-input-41-f78f499b0606> in <module>()
8 with tf.Session() as sess:
9 sess.run(tf.global_variables_initializer())
---> 10 _,c = sess.run([optimizer,cost], feed_dict=
{X:np.asarray(X_train), Y:np.asarray(y_train)})
11 print(c)
.......
.......
ValueError: setting an array element with a sequence.
Please correct the code if you can.
I got the solution. As mentioned in the answers of many other similar questions, the problem is generally with the shape & type of arrays provided to the feed_dict.
My main focus was only on X:X_train[:,0].reshape(-1,1) but it was correct shape and type. The error was in Y:y_train[:, 0].reshape(-1,1). I could not detect this error because I applied one_hot_encoding on y_train but forgot to use the .toarray() method after tranforming. So the shape of y_train appeared to be correct but actually it was wrong.
As a general suggestion after going through many similar questions, I would say to thoroughly check shapes, types and content of the arrays being fed to the feed_dict.
I am trying to train a 2D neural network using keras. I have a weird error message, "ValueError: setting an array element with a sequence." when I try to use model.fit function in keras. Specifically, the error says that my "tensor_train_labels" is a sequence instead of an array. But my labels are indeed numpy arrays (not a sequence). I am not sure why does keras complain about it ?
I am following this tutorial for building my network
tensor_train_data.shape
#TensorShape([Dimension(209), Dimension(64), Dimension(64), Dimension(3)])
tensor_test_data.shape
#TensorShape([Dimension(50), Dimension(64), Dimension(64), Dimension(3)])
tensor_train_labels = tf.reshape(tensor_train_labels, [209,1])
tensor_test_labels = tf.reshape(tensor_test_labels, [50,1])
batch_size = 10
epochs = 8
model = tf.keras.Sequential()
model.add(tf.keras.layers.Conv2D(32, kernel_size=(3,3), activation='relu',
input_shape=(64, 64, 3)))
model.add(tf.keras.layers.MaxPooling2D(pool_size=(2,2)))
model.add(tf.keras.layers.Dropout(0.25))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(128, activation = 'relu'))
model.add(tf.keras.layers.Dropout(0.5))
model.add(tf.keras.layers.Dense(2, activation = 'softmax'))
model.compile(loss='categorical_crossentropy', optimizer =
tf.keras.optimizers.Adam(lr=0.0001, decay=1e-6), metrics=['accuracy'])
model.fit(tensor_train_data/255.0,
tf.keras.utils.to_categorical(tensor_train_labels),
batch_size = batch_size,
shuffle = True,
epochs = epochs,
validation_data = (tensor_test_data/ 255.0,
tf.keras.utils.to_categorical(tensor_test_labels)))
scores = model.evaluate(tensor_test_labels/ 255.0,
tf.keras.utils.to_categorical(tensor_test_labels))
print('Loss: %.3f' % scores[0])
print('Accuracy: %.3f' % scores[1])
The Error :
ValueError Traceback (most recent call last)
<ipython-input-224-80431a1b3e79> in <module>
1 model.compile(loss='categorical_crossentropy', optimizer = tf.keras.optimizers.Adam(lr=0.0001, decay=1e-6), metrics=['accuracy'])
----> 2 model.fit(tensor_train_data/255.0, tf.keras.utils.to_categorical(tensor_train_labels),
3 batch_size = batch_size,
4 shuffle = True,
5 epochs = epochs,
~\AppData\Local\conda\conda\envs\deeplearning\lib\site-packages\tensorflow\python\keras\utils\np_utils.py in to_categorical(y,
num_classes)
37 last.
38 """
---> 39 y = np.array(y, dtype='int')
40 input_shape = y.shape
41 if input_shape and input_shape[-1] == 1 and len(input_shape) > 1:
ValueError: setting an array element with a sequence.
The possible error is that you have arrays of different sizes when you are trying to convert it into the numpy array. Possible solution : https://stackoverflow.com/a/49617425/8185479