I ran my model yesterday on Google colab using their GPU and no error was encountered.
However, today I tried to run the same code (unchanged from yesterday) and I get the following error:
UnknownError: Graph execution error:
Fail to find the dnn implementation.
[[{{node CudnnRNN}}]]
[[sequential/bidirectional/forward_lstm/PartitionedCall]] [Op:__inference_train_function_6026]
The following is my model:
model.add(Embedding(vocab_size,embedding_vector_length,input_length=X_train.shape[1]))
model.add(Bidirectional(LSTM(250,return_sequences=True)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(128,activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(64,activation='relu'))
model.add(Dense(32,activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(16,activation='relu'))
model.add(Dense(num_classes,activation='softmax')) #sigmoid or relu-
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']) #if relu, use mse or mae #maybe oversample lower classes
callbacks = [EarlyStopping(monitor='val_loss', patience=3),ModelCheckpoint('../model/model.h5', save_best_only=True,save_weights_only=False)]
history = model.fit(X_train, y_train, validation_split=0.11, epochs=15, batch_size=32, verbose=1, callbacks=callbacks) #epochs will stop before 15 due to callbacks
Error occurs at this line:
history = model.fit(X_train, y_train, validation_split=0.11, epochs=15, batch_size=32, verbose=1, callbacks=callbacks) #epochs will stop before 15 due to callbacks
I tried solutions from other threads but none of them worked.
Funny enough, it doesn't happen when I use CPU.
Related
I was trying to train a model using IMDB data. I am getting expected train accuracy about 96%+ but I am not satisfied with the test accuracy.Now my expectation is to get 90%+ test accuracy on test data. I tried by using several classifier but each time I am getting 84% to 89% accuracy on test data. Here I am going to include some classifiers I already tried. Most of the cases I tried some parameter tuning by increasing epoch or changing the optimizer. Now my concern is how can I increase the test accuracy to 90%+ .
Classifiers I tried so far:
First:
model = Sequential()
model.add(Embedding(vocab_size, 32, input_length = max_words))
model.add(Bidirectional(LSTM(32, return_sequences = True)))
model.add(GlobalMaxPool1D())
model.add(Dense(20, activation="relu"))
model.add(Dropout(0.05))
model.add(Dense(1, activation="sigmoid"))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(x_train,y_train,validation_data=(x_test, y_test),epochs=10,batch_size=100)
Second:
model = Sequential([
Embedding(vocab_size, 32, input_length=max_words),
Dropout(0.2),
ZeroPadding1D(padding=1),
Convolution1D(64, 5, activation='relu'),
Dropout(0.2),
MaxPooling1D(),
Flatten(),
Dense(100, activation='relu'),
Dropout(0.2),
Dense(1, activation='sigmoid')
])
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(x_train,y_train,validation_data=(x_test, y_test),epochs=10,batch_size=100)
By checking on State-of-the-art analysis on IMDB dataset, I don't think you can get to ^90% with simple models like those you are using. However, you may try using pretrained embedding like glove instead of training your own embedding. Also, I found this repo have BERT implementation in keras, providing demo of IMBD classification, it is able to get ~99% acc.
I have trained a keras LSTM model. But after training, all i get is the final parameters of the models after training with 10 epochs and batch size=120. How can i get intermediate parameter after a batch keras.
Example: after 120 sample in each batch i can get the intermediate parameter of this step.
I have tried callback method and backend in keras, but i do not know how to get the
'''python
model = Sequential()
model.add(Embedding(max_features, 32))
#model.add(LSTM(32, return_sequences=True, input_shape=(1,texts.shape[0])))
model.add(LSTM(32))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['acc'])
history_ltsm = model.fit(texts_train, y_train, epochs=10, batch_size=120, validation_split=0.2)
'''
I expected the model run step by step based on each batch to show the intermediate parameters, but not the all epochs.
Thank you very much!
I have a model that needs to detect if a plant is dead or alive. It is only predicting one class, that data is imbalanced, but i have used weights to counter the imbalance.
I have looked at loads of questions about this problem, but none seem to work, apparently this problem occurs when overfitting, so I have used dropout. But the model still only predicts one class.
Heres the model:
model=Sequential()
# Convolutional layer / input layer
model.add(Conv2D(60, 5,5, activation='relu', input_shape=np.shape(X[1])))
model.add(MaxPooling2D(pool_size=(3,3)))
model.add(Dropout(0.8))
model.add(Flatten())
model.add(Dropout(0.7))
model.add(Dense(130, activation='relu'))
model.add(Dropout(0.6))
# Output layer
model.add(Dense(2, activation='softmax'))
model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
model.fit(X, y, epochs=6, batch_size=32, class_weight=class_weight, validation_data=(X_test, y_test))
Usually it should predict both classes with 1: a healthy plant and 0:
an unhealthy plant
Since your problem is a binary classification and your output dimensionality is 2, you should change your activation to softmax.
model.add(Dense(2, activation='softmax'))
However, if you want to keep sigmoid just change your output layer units to 1, this way you will output how likely your input is one of the two classes with only one unit.
model.add(Dense(1, activation='sigmoid'))
I am doing a keras implementation of sequence classification using 2 BLSTM and 2 FC layer. My data shape is (2655,219,835) where 219 is steps and 835 is number of features. Training is with 1800 intsances and test is with 855 instances. There are 4 classification groups- 0,1,2,3 and They are converted to vectors using to_categorical function. The network implementation code is below:
model = Sequential()
model.add(Masking(mask_value=0., input_shape=(219,835)))
model.add(Bidirectional(LSTM(128,return_sequences=True)))
model.add(Bidirectional(LSTM(128)))
model.add(Dense(256,activation='relu'))
model.add(Dense(256,activation='relu'))
model.add(Dense(4, activation='softmax'))
adam=keras.optimizers.Adam(lr=0.0001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False)
model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['accuracy'])
history=model.fit(train_data, label_train, epochs=30, validation_split=0.33, batch_size=128, verbose=1)
pred=model.predict(test_data)
Now, the problem is I am doing way too low accuracy than the reported accuracy in the paper. Could you please help me to find if I am making any mistake in here ?
Thanks!
Trying to add validation to model.fit, but whenever I do I get an error:
ValueError: Cannot feed value of shape (6, 4, 10) for Tensor 'lstm_input_1:0', which has shape '(32, 4, 10)'
Model:
data_dim = 10
timesteps = 4
batch_size = 32
model = Sequential()
model.add(LSTM(128, batch_input_shape=(batch_size, timesteps, data_dim), return_sequences=True, stateful=True))
model.add(LSTM(64, return_sequences=True, stateful=True))
model.add(LSTM(32, stateful=True))
model.add(Dense(2, activation='softmax'))
sgd = SGD(lr=0.001, momentum=0.0, decay=0.0, nesterov=False)
model.compile(optimizer=sgd, loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(x_train, y_train, nb_epoch=50, batch_size=batch_size, validation_split=0.5)
What could be the error? If I remove validation_split the training works just fine. I've also tried to manually split my training set into two and add it with validation_data=(x_val, y_val) but I got the exact same error.
The issue comes from the fact that you hard code the batch_size value of your inputs. You have fixed it to 32 and then when you try and validate your model, the validation data is sent with a batch of 6 samples, this might be because you don't have enough validation data or maybe because the number of sample isn't a multiple of 32... However, I would let the batch_size free if I was you. Like this:
model.add(LSTM(128, input_shape=(timesteps, data_dim), return_sequences=True, stateful=True))
You specify input_shape instead of batch_input_shape. That way, your network will accept any size of batch, every layer down in the stream of your model are made to adapt to any batch_size if not hardcoded.
I hope this helps :)