Multi input Functional API CNN model - keras

I am making multi input cnn model using keras functional api but it
is giving error... data:
trainset1 = trainset.flow_from_directory(
'/content/',
target_size=(404,410),
batch_size=32,
#seed=50,
class_mode='categorical') print('In Training Set..Entropy....') trainset12 = trainset.flow_from_directory(
'/content/',
target_size=(404,410),
batch_size=32,
#seed=50,
class_mode='categorical')
model: input1 = Input(shape=(404,410,3)) input2 = Input(shape =
(404,410,3))
# x = layers.Dense(128, activation= 'relu')
x = layers.Conv2D(25, (5, 5), activation='relu',
padding='same')(input1) x = layers.MaxPool2D(pool_size=(2, 2),
padding='same')(x)
x1 = layers.Conv2D(25, (5, 5), activation='relu',
padding='same')(input2) x1 = layers.MaxPool2D(pool_size=(2, 2),
padding='same')(x1) flat_layer1 = Flatten()(x) flat_layer2 =
Flatten()(x1)
print(flat_layer1.shape)
print(flat_layer2.shape) concat_layer= Concatenate()([flat_layer1,flat_layer2])
concat_layer= concatenate([flat_layer1,flat_layer2])
x = layers.Dense(16, activation= 'relu')(flat_layer1) #(concat_layer) outputs = layers.Dense(2, activation='softmax')(concat_layer) model =
keras.Model(inputs=[input1,input2], outputs = outputs)
model.compile(
loss = keras.losses.BinaryCrossentropy(),
optimizer=keras.optimizers.Adam(learning_rate=0.001),
metrics=["accuracy"] )
model.fit([trainset1,trainset12] ,batch_size=32,epochs=5, verbose=2)
GIVING ERROR:
--------------------------------------------------------------------------- ValueError Traceback (most recent
call last) in ()
----> 1 model.fit([trainset1,trainset12] ,batch_size=32,epochs=5, verbose=2)
1 frames
/usr/local/lib/python3.7/dist-packages/keras/engine/data_adapter.py
in select_data_adapter(x, y)
989 "Failed to find data adapter that can handle "
990 "input: {}, {}".format(
--> 991 _type_name(x), _type_name(y)))
992 elif len(adapter_cls) > 1:
993 raise RuntimeError(
ValueError: Failed to find data adapter that can handle input:
(<class 'list'> containing values of types {"<class
'keras.preprocessing.image.DirectoryIterator'>"}), <class
'NoneType'"""
what should i do now?

Related

assertion failed: [Condition x == y did not hold element-wise:]

I have built a BiLSTM model with an attention layer for sentence classification task but I am getting an error that my assertion has failed due to mismatch in number of parameters. The attention layer code is here and the error is below the code.
class attention(Layer):
def __init__(self, return_sequences=True):
self.return_sequences = return_sequences
super(attention,self).__init__()
def build(self, input_shape):
self.W=self.add_weight(name="att_weight", shape=(input_shape[-1],1),
initializer="normal")
self.b=self.add_weight(name="att_bias", shape=(input_shape[1],1),
initializer="zeros")
super(attention,self).build(input_shape)
def call(self, x):
e = K.tanh(K.dot(x,self.W)+self.b)
a = K.softmax(e, axis=1)
output = x*a
if self.return_sequences:
return output
return K.sum(output, axis=1)
When i am training the model with attention layer included, it is giving an error that assertion failed.
Epoch 1/10
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
<ipython-input-45-ac310033130c> in <module>()
1 #Early stopping, Adam, dropout = 0.3, 0.5, 0.5
2 #history = model.fit(sequences_matrix, Y_train, batch_size=256, epochs=5, validation_split=0.1, callbacks=[EarlyStopping(monitor='val_loss', min_delta=0.0001)])
----> 3 history = model.fit(sequences_matrix, Y_train, batch_size=32, epochs=10, validation_split=0.1)
8 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
58 ctx.ensure_initialized()
59 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 60 inputs, attrs, num_outputs)
61 except core._NotOkStatusException as e:
62 if name is not None:
InvalidArgumentError: assertion failed: [Condition x == y did not hold element-wise:] [x (sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/Shape_1:0) = ] [32 1] [y (sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/strided_slice:0) = ] [32 758]
[[node sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/assert_equal_1/Assert/Assert (defined at <ipython-input-45-ac310033130c>:3) ]] [Op:__inference_train_function_19854]
Function call stack:
train_function
My model is
model = Sequential()
model.add(Embedding(max_words, 768, input_length=max_len, weights=[embedding]))
model.add(BatchNormalization())
model.add(Activation('tanh'))
model.add(SpatialDropout1D(0.1))
model.add(Conv1D(16, kernel_size=11, activation='relu'))
model.add(Bidirectional(LSTM(16, return_sequences=True)))
model.add(attention(return_sequences=True))
model.add(BatchNormalization())
model.add(Activation('tanh'))
model.add(Dropout(0.2))
model.add(Dense(2, activation='softmax', use_bias=True, kernel_regularizer=regularizers.l1_l2(l1=1e-5, l2=1e-4), bias_regularizer=regularizers.l2(1e-4),
activity_regularizer=regularizers.l2(1e-5)))
model.summary()
Shape of Y_train is
max_words = 48369
max_len = 768
tok = Tokenizer(num_words = max_words)
tok.fit_on_texts(X_train)
sequences = tok.texts_to_sequences(X_train)
sequences_matrix = sequence.pad_sequences(sequences, maxlen = max_len)
Y_train = np.array(Y_train)
Y_test = np.array(Y_test)
print(Y_train.shape)
(43532, 1)
your target is in 2D so you need to set return_sequences=False in the last attention layer in order to return output in 2D format
Add flatten layer before Dropout and then execute.
model.add(Flatten())

Found input variables with inconsistent numbers of samples: [2, 8382]

my method has 2 input data model
network
branches:
49The first branch is composed of an embedding followed by simple Multi-layer
Perceptron (MLP) designed to handle input of the product description.
The second branch is a CNN to operate over the product image data.
These branches are then be concatenated together to form the final.
The problem is when we try to split the data with train_test_split by Cross Validation, It given as this error.
ValueError: Found input variables with inconsistent numbers of samples: [2, 8382]
MLP and CNN
def create_mlp(dim, regress=False):
# define our MLP network
model = Sequential()
model.add(Dense(8, input_dim=dim, activation="relu"))
model.add(Dense(4, activation="relu"))
# check to see if the regression node should be added
if regress:
model.add(Dense(1, activation="linear"))
# return our model
return model
def create_cnn(width, height, depth, filters=(64, 32, 16), regress=False):
# initialize the input shape and channel dimension, assuming
# TensorFlow/channels-last ordering
inputShape = (height, width, depth)
chanDim = -1
# define the model input
inputs = Input(shape=inputShape)
# loop over the number of filters
for (i, f) in enumerate(filters):
# if this is the first CONV layer then set the input
# appropriately
if i == 0:
x = inputs
# CONV => RELU => BN => POOL
x = Conv2D(f, (3, 3), padding="same")(x)
x = Activation("relu")(x)
x = BatchNormalization(axis=chanDim)(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
# flatten the volume, then FC => RELU => BN => DROPOUT
x = Flatten()(x)
x = Dense(16)(x)
x = Activation("relu")(x)
x = BatchNormalization(axis=chanDim)(x)
x = Dropout(0.5)(x)
# apply another FC layer, this one to match the number of nodes
# coming out of the MLP
x = Dense(4)(x)
x = Activation("relu")(x)
# check to see if the regression node should be added
if regress:
x = Dense(1, activation="linear")(x)
# construct the CNN
model = Model(inputs, x)
# return the CNN
return model
mlp = create_mlp(trainEmbedX.shape[1], regress=False)
cnn = create_cnn(64, 64, 3, regress=False)
combinedInput = concatenate([mlp.output, cnn.output])
x = Dense(4, activation="relu")(combinedInput)
x = Dense(1, activation="sigmoid")(x)
model = Model(inputs=[mlp.input, cnn.input], outputs=x)
model.compile(loss="binary_crossentropy", metrics=['accuracy'], optimizer="adam") # binary_crossentropy
The error occurs here
n_folds=3
epochs=3
batch_size=128
#save the model history in a list after fitting so that we can plot later
model_history = []
for i in range(n_folds):
print("Training on Fold: ",i+1)
t_x, val_x, t_y, val_y = train_test_split([trainEmbedX,trainImagesX], trainY, test_size = 0.2, random_state = np.random.randint(1,1000, 1)[0])
model_history.append(fit_and_evaluate(t_x, val_x, t_y, val_y, epochs, batch_size))
print("======="*12, end="\n\n\n")
Training on Fold: 1
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-84-651638774259> in <module>
7 for i in range(n_folds):
8 print("Training on Fold: ",i+1)
----> 9 t_x, val_x, t_y, val_y = train_test_split([trainEmbedX,trainImagesX], trainY, test_size = 0.2, random_state = np.random.randint(1,1000, 1)[0])
10 model_history.append(fit_and_evaluate(t_x, val_x, t_y, val_y, epochs, batch_size))
11 print("======="*12, end="\n\n\n")
~/anaconda3/envs/baron/lib/python3.6/site-packages/sklearn/model_selection/_split.py in train_test_split(*arrays, **options)
2182 test_size = 0.25
2183
-> 2184 arrays = indexable(*arrays)
2185
2186 if shuffle is False:
~/anaconda3/envs/baron/lib/python3.6/site-packages/sklearn/utils/validation.py in indexable(*iterables)
258 else:
259 result.append(np.array(X))
--> 260 check_consistent_length(*result)
261 return result
262
~/anaconda3/envs/baron/lib/python3.6/site-packages/sklearn/utils/validation.py in check_consistent_length(*arrays)
233 if len(uniques) > 1:
234 raise ValueError("Found input variables with inconsistent numbers of"
--> 235 " samples: %r" % [int(l) for l in lengths])
236
237
ValueError: Found input variables with inconsistent numbers of samples: [2, 8382]
This error happens with mismatching dimensions of X and Y in train_test_split.
By looking at your snippet, you try to concatenate two arrays by [trainEmbedX,trainImagesX] which will add a dimension if the original arrays trainEmbedX and trainImagesX are not 1D, hence you have the shape [2, 8382] in the error.
So instead of [trainEmbedX,trainImagesX], I suggest to use np.concatenate to merge these two arrays by np.concatenate((trainEmbedX,trainImagesX),axis=1).

TensorFlow Federated - Adapting existing keras model

I'm having trouble adapting an existing Keras model to work with TenforFlow Federated.
The existing model is a 1D convolutional autoencoder (details shown below)
Existing Model:
input_window = Input(shape=(window_length,1))
x = Conv1D(16, 3, activation="relu", padding="same")(input_window)
x = MaxPooling1D(2, padding="same")(x)
x = Conv1D(1, 3, activation="relu", padding="same")(x)
encoded = MaxPooling1D(2, padding="same")(x)
encoder = Model(input_window, encoded)
x = Conv1D(1, 3, activation="relu", padding="same")(encoded)
x = UpSampling1D(2)(x)
x = Conv1D(16, 1, activation='relu')(x)
x = UpSampling1D(2)(x)
decoded = Conv1D(1, 3, activation='sigmoid', padding='same')(x)
autoencoder = Model(input_window, decoded)
Training data is passed as a numpy.ndarray of shape (102, 48, 1).
Conceptually, this represents 102 days worth of data, each containg 48 values. I can provide an example of this if it would assist in answering.
My attempt to convert the model is shown below.
Converted Model:
def create_compiled_keras_model():
input_window = tf.keras.layers.Input(shape=(window_length,1))
x = tf.keras.layers.Conv1D(16, 3, activation="relu", padding="same")(input_window)
x = tf.keras.layers.MaxPooling1D(2, padding="same")(x)
x = tf.keras.layers.Conv1D(1, 3, activation="relu", padding="same")(x)
encoded = tf.keras.layers.MaxPooling1D(2, padding="same")(x)
encoder = tf.keras.Model(input_window, encoded)
x = tf.keras.layers.Conv1D(1, 3, activation="relu", padding="same")(encoded)
x = tf.keras.layers.UpSampling1D(2)(x)
x = tf.keras.layers.Conv1D(16, 1, activation='relu')(x)
x = tf.keras.layers.UpSampling1D(2)(x)
decoded = tf.keras.layers.Conv1D(1, 3, activation='sigmoid', padding='same')(x)
autoencoder = tf.keras.Model(input_window, decoded)
autoencoder.compile(optimizer='adam', loss='MSE')
return autoencoder
sample_batch = train // numpy.ndarray of shape (102, 48, 1)
def model_fn():
keras_model = create_compiled_keras_model()
return tff.learning.from_compiled_keras_model(keras_model, train)
This produces the error:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-397-9bed171c79be> in <module>
----> 1 model = model_fn()
<ipython-input-396-13bc1955a7f2> in model_fn()
1 def model_fn():
2 keras_model = create_compiled_keras_model()
----> 3 return tff.learning.from_compiled_keras_model(keras_model, train)
~/miniconda3/lib/python3.6/site-packages/tensorflow_federated/python/learning/model_utils.py in from_compiled_keras_model(keras_model, dummy_batch)
190 raise ValueError('`keras_model` must be compiled. Use from_keras_model() '
191 'instead.')
--> 192 return enhance(_TrainableKerasModel(keras_model, dummy_batch))
193
194
~/miniconda3/lib/python3.6/site-packages/tensorflow_federated/python/learning/model_utils.py in __init__(self, inner_model, dummy_batch)
434 # until the model has been called on input. The work-around is to call
435 # Model.test_on_batch() once before asking for metrics.
--> 436 inner_model.test_on_batch(**dummy_batch)
437 # This must occur after test_on_batch()
438 if len(inner_model.loss_functions) != 1:
TypeError: test_on_batch() argument after ** must be a mapping, not numpy.ndarray
So far I have been unable to resolve this.
Is this an issue relating to my model not being compiled correctly, or due to the way I'm passing data?
Any help in resolving this would be greatly appreciated, thanks!
The sample batch should be something that can be passed to batch_input argument of tff.learning.Model.forward_pass.
For wrapped Keras models, this must be a dict with keys matching the arguments to tf.keras.models.Model.test_on_batch.
For this case, I think you may be able to simply wrap the sample batch in a dict with a single key of x:
numpy_sample_batch = train // numpy.ndarray
sample_batch = {'x': numpy_sample_batch}

Proper models import from pytorch in keras

How to import this model from pytorch to keras? I write model from post bottom but Keras and pytorch models give different results.
class net_pytorch(torch.nn.Module):
def __init__(self,Nin=6,Nout=1,Nlinear=112*60):
super(vel_regressor, self).__init__()
self.model1 = torch.nn.Sequential(
torch.nn.Conv1d(Nin,60,kernel_size=3,stride=1,groups=Nin),
torch.nn.ReLU(),
torch.nn.Conv1d(60,120,kernel_size=3,stride=1,groups=Nin),
torch.nn.ReLU(),
torch.nn.Conv1d(120,240,kernel_size=3,stride=1),
torch.nn.ReLU(),
torch.nn.MaxPool1d(10, stride=6),
)
self.model2=model2=torch.nn.Sequential(
torch.nn.Linear(Nlinear, 10*40),
torch.nn.ReLU(),
torch.nn.Linear(10*40, 100),
torch.nn.ReLU(),
torch.nn.Linear(100, Nout)
)
def forward(self, x):
x = self.model1(x)
x = x.view(x.size(0), -1)
x = self.model2(x)
return x
How I write it in keras:
def net_keras():
model2 = Sequential()
model2.add(layers.SeparableConv1D(60, 3, strides=1, activation='relu', depth_multiplier = 6 , name = 'model1.0', input_shape=(200, 6)))
model2.add(layers.SeparableConv1D(120, 3, strides=1, activation='relu', depth_multiplier = 6, name = 'model1.2'))
model2.add(layers.SeparableConv1D(240, 3, strides=1, activation='relu', name = 'model1.4'))
model2.add(layers.GlobalAveragePooling1D())
model2.add(layers.Dense(6720, activation='relu', name = 'model2.0'))
model2.add(layers.Dense(400, activation='relu', name = 'model2.2'))
model2.add(layers.Dense(100, activation='relu', name = 'model2.4'))
model2.add(layers.Dense(3))
model2.compile(optimizer=Adam(), loss='mae')
return model2
I try to use nn-transfer to convert but have error:
Layer names in PyTorch state_dict ['model1.0', 'model1.2', 'model1.4', 'model2.0', 'model2.2', 'model2.4']
Layer names in Keras HDF5 ['dense_1', 'global_average_pooling1d_1', 'model1.0', 'model1.2', 'model1.4', 'model2.0', 'model2.2', 'model2.4']
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-49-a3986379ed2b> in <module>()
----> 1 transfer.pytorch_to_keras(pytorch_network, model2)
/content/nn-transfer/nn_transfer/transfer.py in pytorch_to_keras(pytorch_model, keras_model, flip_filters, flip_channels, verbose)
122 for layer in pytorch_layer_names:
123
--> 124 params = util.dig_to_params(model_weights[layer])
125
126 weight_key = layer + '.weight'
/content/nn-transfer/nn_transfer/util.py in dig_to_params(keras_h5_layer)
23 # ['dense_2']['dense_3']['conv2d_7']['dense_4']['conv1']
24 while not _contains_weights(keras_h5_layer):
---> 25 keras_h5_layer = keras_h5_layer[list(keras_h5_layer.keys())[0]]
26
27 return keras_h5_layer
AttributeError: 'Dataset' object has no attribute 'keys'
Also I try to use pytorch2keras but it dont work with groups != 1.
MMdnn also dont work with this model (error in image).
MMdnn error
Here is the error place :
model.add(layers.Conv1D(60, 3, strides=1, activation='relu', input_shape=(None, 200), name='model1.0'))
You should use input_shape=(None,6) unless your input is always BatchSizex200x6.
In addition, there are tools to convert models between different architectures, e.g. https://github.com/Microsoft/MMdnn.

keras LSTM model input and output dimensions mismatch

model = Sequential()
model.add(Embedding(630, 210))
model.add(LSTM(1024, dropout = 0.2, return_sequences = True))
model.add(LSTM(1024, dropout = 0.2, return_sequences = True))
model.add(Dense(210, activation = 'softmax'))
model.compile(loss = 'categorical_crossentropy', optimizer = 'adam', metrics = ['accuracy'])
filepath = 'ner_2-{epoch:02d}-{loss:.5f}.hdf5'
checkpoint = ModelCheckpoint(filepath, monitor = 'loss', verbose = 1, save_best_only = True, mode = 'min')
callback_list = [checkpoint]
model.fit(X, y , epochs = 20, batch_size = 1024, callbacks = callback_list)
X: the input vector is of the shape (204564, 630, 1)
y: the target vector is of the shape (204564, 210, 1)
i.e. for every 630 inputs 210 outputs have to be predicted but the code throws the following error on compilation
ValueError Traceback (most recent call last)
<ipython-input-57-05a6affb6217> in <module>()
50 callback_list = [checkpoint]
51
---> 52 model.fit(X, y , epochs = 20, batch_size = 1024, callbacks = callback_list)
53 print('successful')
ValueError: Error when checking model input: expected embedding_8_input to have 2 dimensions, but got array with shape (204564, 630, 1)
Please someone explain why this error is occurring and how to solve this
The message says:
Your first layer expects an input with 2 dimensions: (BatchSize, SomeOtherDimension). But your input has 3 dimensions (BatchSize=204564,SomeOtherDimension=630, 1).
Well... remove the 1 from your input, or reshape it inside the model:
Solution 1 - Removing it from the input:
X = X.reshape((204564,630))
Solution 2 - Adding a reshape layer:
model = Sequential()
model.add(Reshape((630,),input_shape=(630,1)))
model.add(Embedding.....)

Resources