I am working on time series prediction using RNNs implemented in Keras on Google Colaboratory. I implemented the RNN as follows:
from tensorflow import keras
mae = keras.losses.MeanAbsoluteError()
hidden_neurons = 50
output_neurons = 1
epoch_size = 50
batch_size = 72
# x_train has shape (500, 1, 23)
LSTM_layer = keras.layers.LSTM(hidden_neurons, input_shape = (x_train.shape[1], x_train.shape[2]), dropout = 0.05)
output_layer = keras.layers.Dense(1)
test_model = keras.Sequential(layers = (LSTM_layer, output_layer))
test_model.reset_states()
test_model.compile(optimizer = 'adam', loss = mae)
test_model.summary()
history = test_model.fit(tf.expand_dims(x_train, axis=-1), y_train, epochs = epoch_size, batch_size = batch_size, validation_data=(x_test, y_test), shuffle = False)
# y_train has shape (500, 1)
# x_test has shape (500, 1, 23)
# y_test has shape (500, 1)
I have the above code (except the import) in a single code cell. When I start a fresh runtime, the network trains fine as expected. But after executing the code cell for around 3-4 times, Colab throws the following error:
ValueError Traceback (most recent call last)
<ipython-input-23-3ac5cc808611> in <module>
12 test_model.compile(optimizer = 'adam', loss = mae)
13 test_model.summary()
---> 14 history = test_model.fit(tf.expand_dims(x_train, axis=-1), y_train, epochs = epoch_size, batch_size = batch_size, validation_data=(x_test, y_test), shuffle = False)
...
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py in autograph_handler(*args, **kwargs)
1145 except Exception as e: # pylint:disable=broad-except
1146 if hasattr(e, "ag_error_metadata"):
-> 1147 raise e.ag_error_metadata.to_exception(e)
1148 else:
1149 raise
ValueError: Input 0 of layer "sequential_2" is incompatible with the layer: expected shape=(None, 1, 23), found shape=(None, 23)
The error persists if tf.expand_dims(x_train, axis=-1)) is omitted in test_model.fit() while fitting the Sequential model.
I guess this has something to do with the layer inputs somehow being changed during execution. I have tried using test_model.reset_states() and running
keras.backend.clear_session()
del test_model
in a separate code cell, but only forcibly killing the Colab runtime seems to work:
import os
os.kill(os.getpid(), 9)
What could cause the layer inputs to change midway during program run?
EDIT: I got the same error when I tried running the cells on Jupyter Notebook on my PC rather than on Colab.
Related
I am working on a multi-label image classification problem with 13 labels. I want to use Hamming Loss to evaluate the performance of the model. So I specified tfa.metrics.HammingLoss(mode = 'multilabel') in the metrics parameter during model compilation. This worked when I provided both X_train and y_train to model.fit(), but it threw a ValueError when I used a Sequence object (described below) for training.
Data Generator description
I used a keras.utils.Sequence input object similar to what is present here. The generator returns 2 numpy arrays for each batch - the first array consists of the input images of shape (128, 128, 3) and the second array consists of labels each of shape (13,).
This is what my code looks like:
model.compile(
loss='binary_crossentropy',
optimizer='rmsprop',
metrics=[tfa.metrics.HammingLoss(mode = 'multilabel')]
)
model.fit(
train_datagen,
epochs = 5,
batch_size = BATCH_SIZE,
steps_per_epoch = TOTAL // BATCH_SIZE
)
And this is the error that I obtained:
Epoch 1/5
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-140-978987a2bbaa> in <module>
3 epochs=5,
4 batch_size=BATCH_SIZE,
----> 5 steps_per_epoch = 2000 // BATCH_SIZE
6 # validation_data=validation_generator,
7 )
4 frames
/usr/local/lib/python3.7/dist-packages/tensorflow_addons/metrics/hamming.py in else_body_2()
64 try:
65 do_return = True
---> 66 retval_ = (ag__.ld(nonzero) / ag__.converted_call(ag__.ld(y_true).get_shape, (), None, fscope)[(- 1)])
67 except:
68 do_return = False
ValueError: in user code:
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1051, in train_function *
return step_function(self, iterator)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_addons/metrics/utils.py", line 66, in update_state *
matches = self._fn(y_true, y_pred, **self._fn_kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_addons/metrics/hamming.py", line 133, in hamming_loss_fn *
return nonzero / y_true.get_shape()[-1]
ValueError: None values not supported.
How do I correct this? Is there any issue with the format of the labels?
I have built a BiLSTM model with an attention layer for sentence classification task but I am getting an error that my assertion has failed due to mismatch in number of parameters. The attention layer code is here and the error is below the code.
class attention(Layer):
def __init__(self, return_sequences=True):
self.return_sequences = return_sequences
super(attention,self).__init__()
def build(self, input_shape):
self.W=self.add_weight(name="att_weight", shape=(input_shape[-1],1),
initializer="normal")
self.b=self.add_weight(name="att_bias", shape=(input_shape[1],1),
initializer="zeros")
super(attention,self).build(input_shape)
def call(self, x):
e = K.tanh(K.dot(x,self.W)+self.b)
a = K.softmax(e, axis=1)
output = x*a
if self.return_sequences:
return output
return K.sum(output, axis=1)
When i am training the model with attention layer included, it is giving an error that assertion failed.
Epoch 1/10
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
<ipython-input-45-ac310033130c> in <module>()
1 #Early stopping, Adam, dropout = 0.3, 0.5, 0.5
2 #history = model.fit(sequences_matrix, Y_train, batch_size=256, epochs=5, validation_split=0.1, callbacks=[EarlyStopping(monitor='val_loss', min_delta=0.0001)])
----> 3 history = model.fit(sequences_matrix, Y_train, batch_size=32, epochs=10, validation_split=0.1)
8 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
58 ctx.ensure_initialized()
59 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 60 inputs, attrs, num_outputs)
61 except core._NotOkStatusException as e:
62 if name is not None:
InvalidArgumentError: assertion failed: [Condition x == y did not hold element-wise:] [x (sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/Shape_1:0) = ] [32 1] [y (sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/strided_slice:0) = ] [32 758]
[[node sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/assert_equal_1/Assert/Assert (defined at <ipython-input-45-ac310033130c>:3) ]] [Op:__inference_train_function_19854]
Function call stack:
train_function
My model is
model = Sequential()
model.add(Embedding(max_words, 768, input_length=max_len, weights=[embedding]))
model.add(BatchNormalization())
model.add(Activation('tanh'))
model.add(SpatialDropout1D(0.1))
model.add(Conv1D(16, kernel_size=11, activation='relu'))
model.add(Bidirectional(LSTM(16, return_sequences=True)))
model.add(attention(return_sequences=True))
model.add(BatchNormalization())
model.add(Activation('tanh'))
model.add(Dropout(0.2))
model.add(Dense(2, activation='softmax', use_bias=True, kernel_regularizer=regularizers.l1_l2(l1=1e-5, l2=1e-4), bias_regularizer=regularizers.l2(1e-4),
activity_regularizer=regularizers.l2(1e-5)))
model.summary()
Shape of Y_train is
max_words = 48369
max_len = 768
tok = Tokenizer(num_words = max_words)
tok.fit_on_texts(X_train)
sequences = tok.texts_to_sequences(X_train)
sequences_matrix = sequence.pad_sequences(sequences, maxlen = max_len)
Y_train = np.array(Y_train)
Y_test = np.array(Y_test)
print(Y_train.shape)
(43532, 1)
your target is in 2D so you need to set return_sequences=False in the last attention layer in order to return output in 2D format
Add flatten layer before Dropout and then execute.
model.add(Flatten())
'''
I am using SHAP, for my model analysis, and while calling 'DeepExplainer' I am getting "AttributeError: 'Sequential' object has no attribute 'eval' "
I am using theano with keras instead of tensor-flow, because there is some version mismatch issue with SHAP, same i have posted in other question. So now i am trying same stuff but this time, _Backend i am using PyTorch and model building is fine but while using SHAP DeepExplainer it is throwing Attribute error, i am newbee to this type of error for Model Explainer domain
'''
Input:
print('Pad sequences (samples x time)')
x_train = sequence.pad_sequences(x_train, maxlen=maxlen)
x_test = sequence.pad_sequences(x_test, maxlen=maxlen)
print('x_train shape:', x_train.shape)
print('x_test shape:', x_test.shape)
print('Build model...')
model = Sequential()
model.add(Embedding(max_features, 128))
model.add(LSTM(128, dropout=0.2, recurrent_dropout=0.2))
model.add(Dense(1, activation='sigmoid'))
# try using different optimizers and different optimizer configs
model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
print('Train...')
model.fit(x_train, y_train,
batch_size=batch_size,
epochs=1,
validation_data=(x_test, y_test))
score, acc = model.evaluate(x_test, y_test,
batch_size=batch_size)
print('Test score:', score)
print('Test accuracy:', acc)
output: No Error
After That:
Input:
import shap
# we use the first 100 training examples as our background dataset to integrate over
explainer = shap.DeepExplainer(model, x_train[:100])
Error:
AttributeError Traceback (most recent call last)
<ipython-input-12-9cca779d01d2> in <module>
1 # we use the first 100 training examples as our background dataset to integrate over
----> 2 explainer = shap.DeepExplainer(model,1)
c:\users\shubh\.conda\envs\pytorch_cpu\lib\site-packages\shap\explainers\deep\__init__.py in __init__(self, model, data, session, learning_phase_flags)
79 self.explainer = TFDeepExplainer(model, data, session, learning_phase_flags)
80 elif framework == 'pytorch':
---> 81 self.explainer = PyTorchDeepExplainer(model, data)
82
83 self.expected_value = self.explainer.expected_value
c:\users\shubh\.conda\envs\pytorch_cpu\lib\site-packages\shap\explainers\deep\deep_pytorch.py in __init__(self, model, data)
47 self.target_handle.remove()
48 del self.layer.target_input
---> 49 self.model = model.eval()
50
51 self.multi_output = False
AttributeError: 'Sequential' object has no attribute 'eval'
Any help or direction for resolving this error?
I am trying to train a 2D neural network using keras. I have a weird error message, "ValueError: setting an array element with a sequence." when I try to use model.fit function in keras. Specifically, the error says that my "tensor_train_labels" is a sequence instead of an array. But my labels are indeed numpy arrays (not a sequence). I am not sure why does keras complain about it ?
I am following this tutorial for building my network
tensor_train_data.shape
#TensorShape([Dimension(209), Dimension(64), Dimension(64), Dimension(3)])
tensor_test_data.shape
#TensorShape([Dimension(50), Dimension(64), Dimension(64), Dimension(3)])
tensor_train_labels = tf.reshape(tensor_train_labels, [209,1])
tensor_test_labels = tf.reshape(tensor_test_labels, [50,1])
batch_size = 10
epochs = 8
model = tf.keras.Sequential()
model.add(tf.keras.layers.Conv2D(32, kernel_size=(3,3), activation='relu',
input_shape=(64, 64, 3)))
model.add(tf.keras.layers.MaxPooling2D(pool_size=(2,2)))
model.add(tf.keras.layers.Dropout(0.25))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(128, activation = 'relu'))
model.add(tf.keras.layers.Dropout(0.5))
model.add(tf.keras.layers.Dense(2, activation = 'softmax'))
model.compile(loss='categorical_crossentropy', optimizer =
tf.keras.optimizers.Adam(lr=0.0001, decay=1e-6), metrics=['accuracy'])
model.fit(tensor_train_data/255.0,
tf.keras.utils.to_categorical(tensor_train_labels),
batch_size = batch_size,
shuffle = True,
epochs = epochs,
validation_data = (tensor_test_data/ 255.0,
tf.keras.utils.to_categorical(tensor_test_labels)))
scores = model.evaluate(tensor_test_labels/ 255.0,
tf.keras.utils.to_categorical(tensor_test_labels))
print('Loss: %.3f' % scores[0])
print('Accuracy: %.3f' % scores[1])
The Error :
ValueError Traceback (most recent call last)
<ipython-input-224-80431a1b3e79> in <module>
1 model.compile(loss='categorical_crossentropy', optimizer = tf.keras.optimizers.Adam(lr=0.0001, decay=1e-6), metrics=['accuracy'])
----> 2 model.fit(tensor_train_data/255.0, tf.keras.utils.to_categorical(tensor_train_labels),
3 batch_size = batch_size,
4 shuffle = True,
5 epochs = epochs,
~\AppData\Local\conda\conda\envs\deeplearning\lib\site-packages\tensorflow\python\keras\utils\np_utils.py in to_categorical(y,
num_classes)
37 last.
38 """
---> 39 y = np.array(y, dtype='int')
40 input_shape = y.shape
41 if input_shape and input_shape[-1] == 1 and len(input_shape) > 1:
ValueError: setting an array element with a sequence.
The possible error is that you have arrays of different sizes when you are trying to convert it into the numpy array. Possible solution : https://stackoverflow.com/a/49617425/8185479
The input element has 3 rows each having 199 columns and the output has 46 rows and 1 column
Input.shape, output.shape
((204563, 3, 199), (204563, 46, 1))
When the input is given the following error is thrown:
from keras.layers import Dense
from keras.models import Sequential
from keras.layers.recurrent import SimpleRNN
model = Sequential()
model.add(SimpleRNN(100, input_shape = (Input.shape[1], Input.shape[2])))
model.add(Dense(output.shape[1], activation = 'softmax'))
model.compile(loss = 'categorical_crossentropy', optimizer = 'adam', metrics = ['accuracy'])
model.fit(Input, output, epochs = 20, batch_size = 200)
error thrown:
Epoch 1/20
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-134-378dd431cf45> in <module>()
3 model.add(Dense(y_target.shape[1], activation = 'softmax'))
4 model.compile(loss = 'categorical_crossentropy', optimizer = 'adam', metrics = ['accuracy'])
----> 5 model.fit(X_input, y_target, epochs = 20, batch_size = 200)
.
.
.
ValueError: Error when checking model target: expected dense_6 to have 2 dimensions, but got array with shape (204563, 46, 1)
Please explain the reason for the problem and possible soution
The problem is that SimpleRNN(100) returns a tensor of shape (204563, 100), hence, the Dense(46) (since output.shape[1]=46) will return a tensor of shape (204563, 46), but your y_target have shape (204563, 46, 1). You need to remove the last dimension with, for example, y_target = np.squeeze(y_target), so that the dimension are consistent