I have the following data:
overall reviewTime reviewerID \
0 4 08 24, 2010 u04428712
1 5 10 31, 2009 u06946603
2 4 10 13, 2015 u92735614
3 5 06 28, 2017 u35112935
4 4 10 12, 2015 u07141505
reviewText \
0 So is Katy Perry's new album "Teenage Dream" c...
1 I got this CD almost 10 years ago, and given t...
2 I REALLY enjoy this pairing of Anderson and Po...
3 Finally got it . It was everything thought it ...
4 Look at all star cast. Outstanding record, pl...
summary unixReviewTime \
0 Amazing that I Actually Bought This...More Ama... 1282608000
1 Excellent album 1256947200
2 Love the Music, Hate the Light Show 1444694400
3 Great 1498608000
4 Love these guys. 1444608000
category price itemID reviewHash image
0 Pop $35.93 p70761125 85559980 NaN
1 Alternative Rock $11.28 p85427891 41699565 NaN
2 Pop $89.86 p82172532 24751194 NaN
3 Pop $11.89 p15255251 22820631 NaN
4 Jazz $15.24 p82618188 53377470 NaN
Index(['overall', 'reviewTime', 'reviewerID', 'reviewText', 'summary',
'unixReviewTime', 'category', 'price', 'itemID', 'reviewHash', 'image'],
dtype='object')
(200000, 11)
and shape of data:
X train shape (160000, 100)
X test shape (40000, 100)
y train shape (160000, 5)
y test shape (40000, 5)
Code for modelling:
# Add sequential model
model = Sequential()
# Add embedding layer
# No of output dimenstions is 100 as we embedded with Glove 100d
Embed_Layer = Embedding(vocab_size, 100, weights=[embedding_matrix], input_length=(MAX_SEQUENCE_LENGTH,), trainable=True)
#define Inputs
review_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype= 'int32', name = 'review_input')
review_embedding = Embed_Layer(review_input)
Flatten_Layer = Flatten()
review_flatten = Flatten_Layer(review_embedding)
output_size = 2
dense1 = Dense(100,activation='relu')(review_flatten)
dense2 = Dense(32,activation='relu')(dense1)
predict = Dense(2,activation='softmax')(dense2)
model = Model(inputs=[review_input],outputs=[predict])
model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['acc'])
print(model.summary())
Out:
Model: "functional_33"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
review_input (InputLayer) [(None, 100)] 0
_________________________________________________________________
embedding_17 (Embedding) (None, 100, 100) 22228800
_________________________________________________________________
flatten_17 (Flatten) (None, 10000) 0
_________________________________________________________________
dense_48 (Dense) (None, 100) 1000100
_________________________________________________________________
dense_49 (Dense) (None, 32) 3232
_________________________________________________________________
dense_50 (Dense) (None, 2) 66
=================================================================
Total params: 23,232,198
Trainable params: 23,232,198
Non-trainable params: 0
_________________________________________________________________
None
After y_train = tf.one_hot(y_train, 5), then during the fitting of the model using model.fit(X_train, y_train, epochs= 5, batch_size=32, verbose=True, validation_data=(X_test, y_test)), the following error occurs:
ValueError: in user code:
/home/x/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:806 train_function *
return step_function(self, iterator)
/home/x/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:796 step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
/home/x/.local/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:1211 run
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/home/x/.local/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:2585 call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
/home/x/.local/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:2945 _call_for_each_replica
return fn(*args, **kwargs)
/home/x/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:789 run_step **
outputs = model.train_step(data)
/home/x/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:748 train_step
loss = self.compiled_loss(
/home/x/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/compile_utils.py:204 __call__
loss_value = loss_obj(y_t, y_p, sample_weight=sw)
/home/x/.local/lib/python3.8/site-packages/tensorflow/python/keras/losses.py:149 __call__
losses = ag_call(y_true, y_pred)
/home/x/.local/lib/python3.8/site-packages/tensorflow/python/keras/losses.py:253 call **
return ag_fn(y_true, y_pred, **self._fn_kwargs)
/home/x/.local/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py:201 wrapper
return target(*args, **kwargs)
/home/x/.local/lib/python3.8/site-packages/tensorflow/python/keras/losses.py:1535 categorical_crossentropy
return K.categorical_crossentropy(y_true, y_pred, from_logits=from_logits)
/home/x/.local/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py:201 wrapper
return target(*args, **kwargs)
/home/x/.local/lib/python3.8/site-packages/tensorflow/python/keras/backend.py:4687 categorical_crossentropy
target.shape.assert_is_compatible_with(output.shape)
/home/x/.local/lib/python3.8/site-packages/tensorflow/python/framework/tensor_shape.py:1134 assert_is_compatible_with
raise ValueError("Shapes %s and %s are incompatible" % (self, other))
ValueError: Shapes (32, 5, 5) and (32, 2) are incompatible
Can someone assists with an idea of how to solve that?
If you are trying to classify between 5 classes you shoud use 5 nodes in the last dense layer:
predict = Dense(5,activation='softmax')(dense2)
The model output shape is (None, 2). Try:
y_train = tf.one_hot(y_train, 2)
Related
I'm trying to run a script using my data, but I get this error:
Input 0 of layer "dense_14" is incompatible with the layer: expected axis -1 of input shape to have value 148, but received input with shape (32, 21).
I understand that the shapes of my data which are ((11598, 1)) aren't compatible with the model shapes. If anyone can help ?
This is the code:
from __future__ import print_function, division
import tensorflow as tf
from keras.layers import Input, Dense, Activation
from keras.layers import Maximum, Concatenate
from keras.models import Model
from keras.models import Sequential
from keras.layers import Dense, Dropout, LSTM, BatchNormalization
from keras.callbacks import TensorBoard
from keras.callbacks import ModelCheckpoint
from tensorflow.keras.optimizers import Adam
from sklearn.ensemble import RandomForestClassifier
from sklearn.neural_network import MLPClassifier
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
class MalGAN():
def __init__(self):
self.apifeature_dims = 128
self.z_dims = 20
self.hide_layers = 256
self.generator_layers = [self.apifeature_dims+self.z_dims, self.hide_layers, self.apifeature_dims]
self.substitute_detector_layers = [self.apifeature_dims, self.hide_layers, 1]
self.blackbox = 'MLP'
optimizer = Adam(lr=0.001)
# Build and Train blackbox_detector
self.blackbox_detector = self.build_blackbox_detector()
# Build and compile the substitute_detector
self.substitute_detector = self.build_substitute_detector()
self.substitute_detector.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy'])
# Build the generator
self.generator = self.build_generator()
# The generator takes malware and noise as input and generates adversarial malware examples
#example = Input(shape=(None,1),name='exampe',dtype='float32')
# noise = Input(shape=(None,1),name='noise',dtype='float32')
example = Input(shape=(self.apifeature_dims,))
noise = Input(shape=(self.z_dims,))
input = [example, noise]
malware_examples = self.generator(input)
# For the combined model we will only train the generator
self.substitute_detector.trainable = True
# The discriminator takes generated images as input and determines validity
validity = self.substitute_detector(malware_examples)
# The combined model (stacked generator and substitute_detector)
# Trains the generator to fool the discriminator
self.combined = Model(input, validity)
self.combined.compile(loss='binary_crossentropy', optimizer=optimizer)
def build_blackbox_detector(self):
if self.blackbox is 'MLP':
blackbox_detector = MLPClassifier(hidden_layer_sizes=(50,), max_iter=10, alpha=1e-4,
solver='sgd', verbose=0, tol=1e-4, random_state=1,
learning_rate_init=.1)
return blackbox_detector
def build_generator(self):
#example = Input(shape=(None,1))
# noise = Input(shape=(None,1))
example = Input(shape=(self.apifeature_dims,))
noise = Input(shape=(self.z_dims,))
#example = Input(shape=(256),name='exampe',dtype='float32')
#noise = Input(shape=(256),name='noise',dtype='float32')
x = Concatenate(axis=1)([example, noise])
for dim in self.generator_layers[1:]:
x = Dense(dim)(x)
x = Activation(activation='sigmoid')(x)
x = Maximum()([example, x])
generator = Model([example, noise], x, name='generator')
generator.summary()
return generator
def build_substitute_detector(self):
input = Input(shape=(self.substitute_detector_layers[0],))
#input = Input(shape=(5,256),dtype='float32')
x = input
for dim in self.substitute_detector_layers[1:]:
x = Dense(dim)(x)
x = Activation(activation='sigmoid')(x)
substitute_detector = Model(input, x, name='substitute_detector')
substitute_detector.summary()
return substitute_detector
def load_data(self, filename):
# data = pd.read_csv("feature_vectors_system calls.csv")
data = np.load(filename)
xmal, ymal, xben, yben = data['arr_0'].reshape(-1,1), data['arr_1'].reshape(-1,1), data['arr_2'].reshape(-1,1), data['arr_3'].reshape(-1,1)
return (xmal, ymal), (xben, yben)
def train(self, epochs, batch_size=128):
# Load the dataset
(xmal, ymal), (xben, yben) = self.load_data('my_data.npz')
xtrain_mal, xtest_mal, ytrain_mal, ytest_mal = train_test_split(xmal, ymal, test_size=0.20)
xtrain_ben, xtest_ben, ytrain_ben, ytest_ben = train_test_split(xben, yben, test_size=0.20)
print("xmal is : ", xmal.shape)
print("ymal is : ", ymal.shape)
print("xben is : ",xben.shape)
print("yben is ", yben.shape)
print("here 1")
# Train blackbox_detctor
self.blackbox_detector.fit(np.concatenate([xmal, xben]),
np.concatenate([ymal, yben]))
ytrain_ben_blackbox = self.blackbox_detector.predict(xtrain_ben)
Original_Train_TRR = self.blackbox_detector.score(xtrain_mal, ytrain_mal)
Original_Test_TRR = self.blackbox_detector.score(xtest_mal, ytest_mal)
print("here 2")
Train_TRR, Test_TRR = [], []
for epoch in range(epochs):
for step in range(1):#range(xtrain_mal.shape[0] // batch_size):
# ---------------------
# Train substitute_detector
# ---------------------
# Select a random batch of malware examples
idx = np.random.randint(0, xtrain_mal.shape[0],batch_size)
xmal_batch = xtrain_mal[idx]
noise = np.random.uniform(0, 1, (batch_size, self.z_dims))
idx = np.random.randint(0, xmal_batch.shape[0], batch_size)
xben_batch = xtrain_ben[idx]
yben_batch = ytrain_ben_blackbox[idx]
# Generate a batch of new malware examples
# noise.shape = (320,2)
#xmal_batch = np.asarray(xmal_batch).astype(np.float32)
#noise = np.asarray(noise).astype(np.float32)
# xmal_batch.shape = (32,1)
# noise.shape= (32,20)
print("xmal is: ", xmal_batch.shape)
print("noise is:", noise.shape)
#xmal_batch = tf.convert_to_tensor(xmal_batch)
#noise = tf.convert_to_tensor(noise)
xmal_batch.shape = (128,1)
noise.shape= (128,20)
#print("xmal is:", xmal)
#print("noise is:", noise)
# xmal_batch = np.asarray(xmal_batch).astype(np.float32)
noise = np.asarray(noise).astype(np.float32)
gen_examples = self.generator.predict([xmal_batch, noise])
# gen_examples = np.vectorize(gen_examples)
# gen_examples = np.array(gen_examples)
# gen_examples = gen_examples.reshape(gen_examples.shape[0],-1)
# print(gen_examples.shape)
gen_examples = gen_examples.reshape(-1,1)
ymal_batch = self.blackbox_detector.predict(np.ones(gen_examples.shape)*(gen_examples > 0.5))
print("ymal shape is:", ymal_batch.shape)
print("gen_examples is:", gen_examples.shape)
# gen_examples.shape = (2688,128)
# Train the substitute_detector
ymal_batch = np.array([0 for _ in range(len(gen_examples)//2)] + [1 for _ in range(len(gen_examples)//2)])
d_loss_real = self.substitute_detector.train_on_batch((gen_examples), ymal_batch)
print("yben shape is:", yben_batch.shape)
print("xben shape is:", xben_batch.shape)
# xben_batch.shape =(1,128)
# yben_batch.shape=(128,1)
print("yben shape is:", yben_batch.shape)
print("xben shape is:", xben_batch.shape)
d_loss_fake = self.substitute_detector.train_on_batch(xben_batch, yben_batch)
d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)
# ---------------------
# Train Generator
# ---------------------
idx = np.random.randint(0, xtrain_mal.shape[0], batch_size)
xmal_batch = xtrain_mal[idx]
noise = np.random.uniform(0, 1, (batch_size, self.z_dims))
# Train the generator
g_loss = self.combined.train_on_batch([xmal_batch, noise], np.zeros((batch_size, 1)))
# Compute Train TRR
noise = np.random.uniform(0, 1, (xtrain_mal.shape[0], self.z_dims))
gen_examples = self.generator.predict([xtrain_mal, noise])
TRR = self.blackbox_detector.score(np.ones(gen_examples.shape) * (gen_examples > 0.5), ytrain_mal)
Train_TRR.append(TRR)
# Compute Test TRR
noise = np.random.uniform(0, 1, (xtest_mal.shape[0], self.z_dims))
gen_examples = self.generator.predict([xtest_mal, noise])
TRR = self.blackbox_detector.score(np.ones(gen_examples.shape) * (gen_examples > 0.5), ytest_mal)
Test_TRR.append(TRR)
# Plot the progress
print("%d [D loss: %f, acc.: %.2f%%] [G loss: %f]" % (epoch, d_loss[0], 100*d_loss[1], g_loss))
print('Original_Train_TRR: {0}, Adver_Train_TRR: {1}'.format(Original_Train_TRR, Train_TRR[-1]))
print('Original_Test_TRR: {0}, Adver_Test_TRR: {1}'.format(Original_Test_TRR, Test_TRR[-1]))
# Plot TRR
plt.figure()
plt.plot(range(epochs), Train_TRR, c='r', label='Training Set', linewidth=2)
plt.plot(range(epochs), Test_TRR, c='g', linestyle='--', label='Validation Set', linewidth=2)
plt.xlabel("Epoch")
plt.ylabel("TRR")
plt.legend()
plt.show()
if __name__ == '__main__':
malgan = MalGAN()
malgan.train(epochs=1000, batch_size=128)
The output is:
Model: "substitute_detector"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_16 (InputLayer) [(None, 128)] 0
dense_12 (Dense) (None, 256) 33024
activation_12 (Activation) (None, 256) 0
dense_13 (Dense) (None, 1) 257
activation_13 (Activation) (None, 1) 0
=================================================================
Total params: 33,281
Trainable params: 33,281
Non-trainable params: 0
_________________________________________________________________
Model: "generator"
__________________________________________________________________________________________________
/usr/local/lib/python3.7/dist-packages/keras/optimizer_v2/adam.py:105: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
super(Adam, self).__init__(name, **kwargs)
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_17 (InputLayer) [(None, 128)] 0 []
input_18 (InputLayer) [(None, 20)] 0 []
concatenate_3 (Concatenate) (None, 148) 0 ['input_17[0][0]',
'input_18[0][0]']
dense_14 (Dense) (None, 256) 38144 ['concatenate_3[0][0]']
activation_14 (Activation) (None, 256) 0 ['dense_14[0][0]']
dense_15 (Dense) (None, 128) 32896 ['activation_14[0][0]']
activation_15 (Activation) (None, 128) 0 ['dense_15[0][0]']
maximum_3 (Maximum) (None, 128) 0 ['input_17[0][0]',
'activation_15[0][0]']
==================================================================================================
Total params: 71,040
Trainable params: 71,040
Non-trainable params: 0
__________________________________________________________________________________________________
xmal is : (11598, 1)
ymal is : (11598, 1)
xben is : (11598, 1)
yben is (11598, 1)
here 1
after which I get
/usr/local/lib/python3.7/dist-packages/sklearn/neural_network/_multilayer_perceptron.py:1109: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
y = column_or_1d(y, warn=True)
/usr/local/lib/python3.7/dist-packages/sklearn/neural_network/_multilayer_perceptron.py:696: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (10) reached and the optimization hasn't converged yet.
ConvergenceWarning,
here 2
xmal is: (128, 1)
noise is: (128, 20)
WARNING:tensorflow:Model was constructed with shape (None, 128) for input KerasTensor(type_spec=TensorSpec(shape=(None, 128), dtype=tf.float32, name='input_17'), name='input_17', description="created by layer 'input_17'"), but it was called on an input with incompatible shape (32, 1).
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-4-937d6995f5c4> in <module>()
240 if __name__ == '__main__':
241 malgan = MalGAN()
--> 242 malgan.train(epochs=1000, batch_size=128)
2 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py in autograph_handler(*args, **kwargs)
1145 except Exception as e: # pylint:disable=broad-except
1146 if hasattr(e, "ag_error_metadata"):
-> 1147 raise e.ag_error_metadata.to_exception(e)
1148 else:
1149 raise
ValueError: in user code:
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1801, in predict_function *
return step_function(self, iterator)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1790, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1783, in run_step **
outputs = model.predict_step(data)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1751, in predict_step
return self(x, training=False)
File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/usr/local/lib/python3.7/dist-packages/keras/engine/input_spec.py", line 249, in assert_input_compatibility
f'Input {input_index} of layer "{layer_name}" is '
ValueError: Exception encountered when calling layer "generator" (type Functional).
Input 0 of layer "dense_14" is incompatible with the layer: expected axis -1 of input shape to have value 148, but received input with shape (32, 21)
Call arguments received:
• inputs=('tf.Tensor(shape=(32, 1), dtype=int64)', 'tf.Tensor(shape=(32, 20), dtype=float32)')
• training=False
• mask=None
I have this code:
EMBEDDING_DIM = 100
MAXLEN = 16
TRUNCATING = 'post'
PADDING = 'post'
OOV_TOKEN = "<OOV>"
MAX_EXAMPLES = 160000
TRAINING_SPLIT = 0.9
# Initialize an empty numpy array with the appropriate size
EMBEDDINGS_MATRIX = np.zeros((VOCAB_SIZE+1, EMBEDDING_DIM))
# Iterate all of the words in the vocabulary and if the vector representation for
# each word exists within GloVe's representations, save it in the EMBEDDINGS_MATRIX array
for word, i in word_index.items():
embedding_vector = GLOVE_EMBEDDINGS.get(word)
if embedding_vector is not None:
EMBEDDINGS_MATRIX[i] = embedding_vector
# Define the model
def create_model(vocab_size, embedding_dim, maxlen, embeddings_matrix):
model = tf.keras.Sequential([
# Set the Embedding layer when using pre-trained embeddings
tf.keras.layers.Embedding(vocab_size+1, embedding_dim, input_length=maxlen, weights=[embeddings_matrix], trainable=False),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Conv1D(64, 6, activation='relu'),
# tf.keras.layers.AveragePooling1D(pool_size=4),
tf.keras.layers.GlobalAveragePooling1D(),
tf.keras.layers.LSTM(64),
tf.keras.layers.Dense(8, activation='relu'),
tf.keras.layers.Dense(1, activation='sigmoid')
])
model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
return model.summary()
model = create_model(VOCAB_SIZE, EMBEDDING_DIM, MAXLEN, EMBEDDINGS_MATRIX)
# Train the model
history = model.fit(train_pad_trunc_seq, train_labels, epochs=20, validation_data=(val_pad_trunc_seq, val_labels))
which brings me this error,
ValueError Traceback (most recent call last)
Input In [26], in <cell line: 2>()
1 # Create your untrained model
----> 2 model = create_model(VOCAB_SIZE, EMBEDDING_DIM, MAXLEN, EMBEDDINGS_MATRIX)
4 # Train the model and save the training history
5 history = model.fit(train_pad_trunc_seq, train_labels, epochs=20, validation_data=(val_pad_trunc_seq, val_labels))
Input In [25], in create_model(vocab_size, embedding_dim, maxlen, embeddings_matrix)
4 def create_model(vocab_size, embedding_dim, maxlen, embeddings_matrix):
5
6 ### START CODE HERE
----> 8 model = tf.keras.Sequential([
9 # This is how you need to set the Embedding layer when using pre-trained embeddings
10 tf.keras.layers.Embedding(vocab_size+1, embedding_dim, input_length=maxlen, weights=[embeddings_matrix], trainable=False),
11 tf.keras.layers.Dropout(0.2),
12 tf.keras.layers.Conv1D(64, 6, activation='relu'),
13 # tf.keras.layers.AveragePooling1D(pool_size=4),
14 tf.keras.layers.GlobalAveragePooling1D(),
15 tf.keras.layers.LSTM(64),
16 tf.keras.layers.Dense(8, activation='relu'),
17 tf.keras.layers.Dense(1, activation='sigmoid')
18 ])
20 model.compile(loss='binary_crossentropy',
21 optimizer='adam',
22 metrics=['accuracy'])
24 ### END CODE HERE
File ~\.conda\envs\tf-gpu\lib\site-packages\tensorflow\python\training\tracking\base.py:629, in no_automatic_dependency_tracking.<locals>._method_wrapper(self, *args, **kwargs)
627 self._self_setattr_tracking = False # pylint: disable=protected-access
628 try:
--> 629 result = method(self, *args, **kwargs)
630 finally:
631 self._self_setattr_tracking = previous_value # pylint: disable=protected-access
File ~\.conda\envs\tf-gpu\lib\site-packages\keras\utils\traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb
File ~\.conda\envs\tf-gpu\lib\site-packages\keras\engine\input_spec.py:214, in assert_input_compatibility(input_spec, inputs, layer_name)
212 ndim = shape.rank
213 if ndim != spec.ndim:
--> 214 raise ValueError(f'Input {input_index} of layer "{layer_name}" '
215 'is incompatible with the layer: '
216 f'expected ndim={spec.ndim}, found ndim={ndim}. '
217 f'Full shape received: {tuple(shape)}')
218 if spec.max_ndim is not None:
219 ndim = x.shape.rank
ValueError: Input 0 of layer "lstm_2" is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 64)
So I need to have an input parameter such as input = (?,?,?) for my LSTM layer instead of (None, 64), but what should it be?
I have also tried to change GlobalAveragePooling1D() to AveragePooling1D(pool_size=4).
It brings up the summary but gives me a different error:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
embedding_3 (Embedding) (None, 16, 100) 12829400
dropout_3 (Dropout) (None, 16, 100) 0
conv1d_3 (Conv1D) (None, 11, 64) 38464
average_pooling1d_1 (Averag (None, 2, 64) 0
ePooling1D)
lstm_3 (LSTM) (None, 64) 33024
dense_6 (Dense) (None, 8) 520
dense_7 (Dense) (None, 1) 9
=================================================================
Total params: 12,901,417
Trainable params: 72,017
Non-trainable params: 12,829,400
AttributeError Traceback (most recent call last)
Input In [24], in <cell line: 5>()
2 model = create_model(VOCAB_SIZE, EMBEDDING_DIM, MAXLEN, EMBEDDINGS_MATRIX)
4 # Train the model and save the training history
----> 5 history = model.fit(train_pad_trunc_seq, train_labels, epochs=20, validation_data=(val_pad_trunc_seq, val_labels))
AttributeError: 'NoneType' object has no attribute 'fit'
Please help?
I haven't used it before, but refer to https://www.tensorflow.org/api_docs/python/tf/keras/layers/GlobalAveragePooling1D. It seems that if the input tensor is of dimension n, then GlobalAveragePooling 1D output a tensor with dimension n-1. GlobalAveragePooling1D do pooling along an axis, so it reduces the dim.
If the output tensor shape after conv1d is (None, 11, 64), then the output for GlobalAveragePooling 1D is (None, 64), which is of dimension 2, not 3, so the first attempt results in an error.
Things are different for AveragePooling1D. It does local average pooling, so the dimension of output tensor is the same as input tensor.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/AveragePooling1D
For the second question, refer to https://www.tensorflow.org/api_docs/python/tf/keras/Model#summary. model.summary() just prints a string summary of the network. (I guess the function return value is None). You should return a model with return model, because class Model has the method fit
I create a custom generator tensorflow.keras.utils.Sequence and tried to fit a simple model.
import tensorflow as tf
import keras
import numpy as np
from tensorflow.keras.utils import Sequence
from keras import Sequential
from keras.layers import InputLayer, Dense
class MyDataGenerator(Sequence):
def __init__(self, df, x_col='filename', y_col='class',
batch_size=32, path='./', num_classes=None, shuffle=True,
dim=(634,513,1), nfft=1024, hstep=256, sr=16000):
self.batch_size = batch_size
self.df = df
self.indices = self.df.index.tolist()
self.num_classes = num_classes
self.path = path
self.shuffle = shuffle
self.x_col = x_col
self.y_col = y_col
self.on_epoch_end()
self.dim = dim
self.nfft = nfft
self.hstep = hstep
self.sr = sr
def __getitem__(self, index):
index = self.index[index * self.batch_size:(index + 1) * self.batch_size]
batch = [self.indices[k] for k in index]
X = np.zeros((self.batch_size, *self.dim), dtype=np.float32)
y = np.zeros((self.batch_size,), dtype=np.uint32)
return X, y
def __len__(self):
return math.ceil(len(self.indices) / self.batch_size)
def on_epoch_end(self):
self.index = np.arange(len(self.indices))
if self.shuffle == True:
np.random.shuffle(self.index)
train_datagen = MyDataGenerator(df_training)
valid_datagen = MyDataGenerator(df_validation, shuffle=False)
x,y = train_datagen[27]
print(x.shape, y.shape, x.dtype, y.dtype, type(x), type(y))
print(y)
1. (32, 634, 513, 1) (32,) float32 uint32 <class 'numpy.ndarray'> <class 'numpy.ndarray'>
2. [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
model = Sequential(name='Test_model')
model.add(InputLayer((634, 513, 1), name='Input'))
model.add(Flatten(name='Flatten'))
model.add(Dense(1, activation='sigmoid', name='Output'))
model.summary()
model.compile(optimizer='adam', loss='binary_crossentropy',
metrics=['accuracy'])
model.fit(
train_datagen,
epochs=1,
verbose=0,
validation_data=valid_datagen
)
And i got ValueError
Model: "Test_model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
Flatten (Flatten) (None, 325242) 0
_________________________________________________________________
Output (Dense) (None, 1) 325243
=================================================================
Total params: 325,243
Trainable params: 325,243
Non-trainable params: 0
_________________________________________________________________
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-203-533c619e3ae0> in <module>()
11 epochs=1,
12 verbose=0,
---> 13 validation_data=valid_datagen
14 )
3 frames
/usr/local/lib/python3.7/dist-packages/keras/engine/data_adapter.py in select_data_adapter(x, y)
976 "Failed to find data adapter that can handle "
977 "input: {}, {}".format(
--> 978 _type_name(x), _type_name(y)))
979 elif len(adapter_cls) > 1:
980 raise RuntimeError(
ValueError: Failed to find data adapter that can handle input: <class '__main__.MyDataGenerator'>, <class 'NoneType'>
Can't figure out why the model doesn't want to train with a generator. The data types are correct. Inputs and outputs too.
If try to do it without a generator, then the model works correctly, but the generator cannot be removed due to the huge dataset.
X = np.zeros((32, 634, 513, 1), dtype=np.float32)
y = np.zeros((32,), dtype=np.uint32)
model = Sequential(name='Test_model')
model.add(InputLayer((634, 513, 1), name='Input'))
model.add(Flatten(name='Flatten'))
model.add(Dense(1, activation='sigmoid', name='Output'))
model.summary()
model.compile(optimizer='adam', loss='binary_crossentropy',
metrics=['accuracy'])
model.fit(
X,y,
epochs=1,
verbose=1,
#validation_data=valid_datagen
)
Model: "Test_model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
Flatten (Flatten) (None, 325242) 0
_________________________________________________________________
Output (Dense) (None, 1) 325243
=================================================================
Total params: 325,243
Trainable params: 325,243
Non-trainable params: 0
_________________________________________________________________
1/1 [==============================] - 14s 14s/step - loss: 0.6931 - accuracy: 1.0000
<keras.callbacks.History at 0x7f74adf34510>
Tensorflow version: 2.5.0
Keras version: 2.4.3
I've recently encountered this when looking at a model's summary.
I was wondering, what's the difference between [(None, 16)] and (None, 16)? Why does the Input layer have such input shape?
Source: model.summary() can't print output shape while using subclass model
The issue is how you are defining the input_shape. A single element tuple in python is actually a scalar value as you can see below -
input_shape0 = 32
input_shape1 = (32)
input_shape2 = (32,)
print(input_shape0, input_shape1, input_shape2)
32 32 (32,)
Since Keras function API Input needs an input shape as a tuple, you will have to pass it in the form of (n,) instead of n
It's weird that you get a square bracket because when I run the exact same code, I get an error.
TypeError Traceback (most recent call last)
<ipython-input-828-b564be68c80d> in <module>
33
34 if __name__ == '__main__':
---> 35 mlp = MLP((16))
36 mlp.summary()
<ipython-input-828-b564be68c80d> in __init__(self, input_shape, **kwargs)
6 super(MLP, self).__init__(**kwargs)
7 # Add input layer
----> 8 self.input_layer = klayers.Input(input_shape)
9
10 self.dense_1 = klayers.Dense(64, activation='relu')
~/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/input_layer.py in Input(shape, batch_size, name, dtype, sparse, tensor, **kwargs)
229 dtype=dtype,
230 sparse=sparse,
--> 231 input_tensor=tensor)
232 # Return tensor including `_keras_history`.
233 # Note that in this case train_output and test_output are the same pointer.
~/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/input_layer.py in __init__(self, input_shape, batch_size, dtype, input_tensor, sparse, name, **kwargs)
89 if input_tensor is None:
90 if input_shape is not None:
---> 91 batch_input_shape = (batch_size,) + tuple(input_shape)
92 else:
93 batch_input_shape = None
TypeError: 'int' object is not iterable
Therefore, the right way to do it (which should fix your model summary as well is as below -
from tensorflow import keras
from tensorflow.keras import layers as klayers
class MLP(keras.Model):
def __init__(self, input_shape=(32,), **kwargs):
super(MLP, self).__init__(**kwargs)
# Add input layer
self.input_layer = klayers.Input(input_shape)
self.dense_1 = klayers.Dense(64, activation='relu')
self.dense_2 = klayers.Dense(10)
# Get output layer with `call` method
self.out = self.call(self.input_layer)
# Reinitial
super(MLP, self).__init__(
inputs=self.input_layer,
outputs=self.out,
**kwargs)
def build(self):
# Initialize the graph
self._is_graph_network = True
self._init_graph_network(
inputs=self.input_layer,
outputs=self.out
)
def call(self, inputs):
x = self.dense_1(inputs)
return self.dense_2(x)
if __name__ == '__main__':
mlp = MLP((16,))
mlp.summary()
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_19 (InputLayer) (None, 16) 0
_________________________________________________________________
dense_8 (Dense) (None, 64) 1088
_________________________________________________________________
dense_9 (Dense) (None, 10) 650
=================================================================
Total params: 1,738
Trainable params: 1,738
Non-trainable params: 0
_________________________________________________________________
I am not able to solve the following error, please accept my apologies if it sounds naive, I am very new to Keras.
The output of the encoder is actually a complex value, so each output is real and imaginary part, input_h1 is also a complex value with real and imaginary parts represented as a vector. I want to multiply both of them.
# Input bits
input_bits1 = Input(shape=(2,))
input_bits2 = Input(shape=(2,))
# Input Channels
input_h1 = Input(shape=(2,))
input_h2 = Input(shape=(2,))
# Concatenate both inputs
input_bits = keras.layers.concatenate([input_bits1, input_bits2], axis=1)
print(input_bits)
# Create Encoder
m1 = Dense(64, activation='relu')(input_bits)
m2 = Dense(128, activation='relu')(m1)
encoded1 = Dense(2, activation='linear')(m2)
# Normalize the encoded value
encoded = Lambda(lambda x: K.l2_normalize(x, axis=1))(encoded1)
# The output of the encoder is actually a complex value, so each output is real and imaginary part,input_h1 is also a complex value with real and imaginary parts represented as a vector. I want to multiply both of them.
# mt1 is the real part of complex number multiplication
mt1 = encoded[:,0:1]*input_h1[:,0:1] - encoded[:,1:2]*input_h1[:,1:2]
print(mt1)
# nt1 is the imaginary part of the complex number multiplication
nt1 = encoded[:,0:1]*input_h1[:,1:2] + encoded[:,1:2]*input_h1[:,0:1]
print(nt1)
# Concatenate real and imaginary parts to feed into the decoder
mnt2 = keras.layers.concatenate([mt1, nt1], axis=1)
print(mnt2)
# Decoder 1
x5 = Dense(1024, activation='relu')(mnt2)
x6 = Dense(512, activation='relu')(x5)
x7 = Dense(64, activation='relu')(x6)
decoded_UP1 = Dense(2, activation='tanh')(x7)
# Decoder 2
a3 = Dense(1024, activation='relu')(mnt2)
a4 = Dense(512, activation='relu')(a3)
a5 = Dense(64, activation='relu')(a4)
decoded_UP2 = Dense(2, activation='tanh')(a5)
decoded = keras.layers.concatenate([decoded_UP1, decoded_UP2], axis=1)
autoencoder = Model([input_bits1, input_bits2, input_h1, input_h2], decoded)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')
autoencoder.summary()
I am getting the following output/error :
AttributeError Traceback (most recent call last)
<ipython-input-9-c3710aa7e060> in <module>()
35 decoded = keras.layers.concatenate([decoded_UP1, decoded_UP2], axis=1)
36
---> 37 autoencoder = Model([input_bits1, input_bits2, input_h1, input_h2], decoded)
38 autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')
39 autoencoder.summary()
AttributeError: 'NoneType' object has no attribute '_inbound_nodes'
To improve the clarity of the code, you should create multiple models for instance one for the encoder and one for the decoder. That way you could have a model.summary() for each:
Example for the Bits Encoder:
from keras.layers import Input, Dense, concatenate
from keras.models import Model
# Input
input_bits1 = Input(shape=(2,))
input_bits2 = Input(shape=(2,))
input_bits = keras.layers.concatenate([input_bits1, input_bits2], axis=1)
# Hidden Layers
encoder_bits_h1 = Dense(64, activation='relu')(input_bits)
encoder_bits_h2 = Dense(128, activation='relu')(encoder_bits_h1)
encoder_bits_h3 = Dense(2, activation='linear')(encoder_bits_h2)
# Create the model
bits_encoder = Model(inputs=[input_bits1, input_bits2], outputs=[encoder_bits_h3])
bits_encoder.summary()
Which returns the bits encoder configuration:
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_5 (InputLayer) (None, 2) 0
__________________________________________________________________________________________________
input_6 (InputLayer) (None, 2) 0
__________________________________________________________________________________________________
concatenate_2 (Concatenate) (None, 4) 0 input_5[0][0]
input_6[0][0]
__________________________________________________________________________________________________
dense_4 (Dense) (None, 64) 320 concatenate_2[0][0]
__________________________________________________________________________________________________
dense_5 (Dense) (None, 128) 8320 dense_4[0][0]
__________________________________________________________________________________________________
dense_6 (Dense) (None, 2) 258 dense_5[0][0]
==================================================================================================
Total params: 8,898
Trainable params: 8,898
Non-trainable params: 0
__________________________________________________________________________________________________