I need to make a model that takes as input a 2D binary matrix: (37,10) for instance and return a real 2D matrix of the same shape as the input one. I erote this code but I am not sure what X (in the output layer) should be equal to.
model=Sequential()
model.add(Dense(32,activation='linear',input_shape=(37,10)))
model.add(Dense(32,activation='linear'))
model.add(Dense(X,activation='linear'))
model.compile(loss='mse',optimizer=Adam(lr=self.learning_rate),metrics=['accuracy'])
Please let me know if you think my model is correct as defined and what to write instead of X
Thank you
I updated your code to get shape of output same as input. We need to add Flatten and Reshape layer at the start and end of the model. In simple, X should be equal to the number of elements in the input_shape.
from tensorflow.keras.layers import Dense, Flatten,Reshape
from tensorflow.keras.models import Sequential
from tensorflow.keras.optimizers import Adam
input_shape=(37,10)
num_elm =input_shape[0]*input_shape[1]
model=Sequential()
model.add(Flatten(input_shape=input_shape))
model.add(Dense(32, activation='linear'))
model.add(Dense(32, activation='linear'))
model.add(Dense(num_elm, activation='linear'))
model.add(Reshape(input_shape))
model.compile(loss='mse',optimizer=Adam(),metrics=['accuracy'])
model.summary()
Model: "sequential_5"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten_4 (Flatten) (None, 370) 0
_________________________________________________________________
dense_14 (Dense) (None, 32) 11872
_________________________________________________________________
dense_15 (Dense) (None, 32) 1056
_________________________________________________________________
dense_16 (Dense) (None, 370) 12210
_________________________________________________________________
reshape (Reshape) (None, 37, 10) 0
=================================================================
Total params: 25,138
Trainable params: 25,138
Non-trainable params: 0
_________________________________________________________________
X will be 10 even though using FC layers for 2-d data may not be very suitable in the first place, also you're sure the metrics will be accuracy.
Here's your model with the correct output shape.
Code:
from tensorflow.keras.layers import *
from tensorflow.keras.models import Model, Sequential
from tensorflow.keras.optimizers import Adam
import tensorflow as tf
import numpy as np
model=Sequential()
model.add(Dense(32,activation='linear',input_shape=(37,10)))
model.add(Dense(32,activation='linear'))
model.add(Dense(10,activation='linear'))
model.compile(loss='mse',optimizer=Adam(lr=.001),metrics=['accuracy'])
model.summary()
Model: "sequential_3"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_8 (Dense) (None, 37, 32) 352
_________________________________________________________________
dense_9 (Dense) (None, 37, 32) 1056
_________________________________________________________________
dense_10 (Dense) (None, 37, 10) 330
=================================================================
Total params: 1,738
Trainable params: 1,738
Non-trainable params: 0
__________________________
Related
I am testing something which includes building a FCNN network Dynamically. Idea is to build Number of layers and it's neurons based on a given list and the dummy code is:
neurons = [10,20,30] # First Dense has 10 neuron, 2nd has 20 and third has 30
inputs = keras.Input(shape=(1024,))
x = Dense(10,activation='relu')(inputs)
for n in neurons:
x = Dense(n,activation='relu')(x)
out = Dense(1,activation='sigmoid')(x)
model = Model(inputs,out)
model.summary()
keras.utils.plot_model(model,'model.png')
for layer in model.layers:
print(layer.name)
To my surprise, it is showing nothing.I even compiled and ran the functions again and nothing came out.
The model.summary always shows number of trainable and non trainable params but not the model structure and layer names. Why is this happening? Or is this normal?
About model.summary(), don't mix tf 2.x and standalone keras at a time. If I ran you model in tf 2.x, I get the expected results.
from tensorflow.keras.layers import *
from tensorflow.keras import Model
from tensorflow import keras
# your code ...
model.summary()
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 1024)] 0
_________________________________________________________________
dense (Dense) (None, 10) 10250
_________________________________________________________________
dense_1 (Dense) (None, 10) 110
_________________________________________________________________
dense_2 (Dense) (None, 20) 220
_________________________________________________________________
dense_3 (Dense) (None, 30) 630
_________________________________________________________________
dense_4 (Dense) (None, 1) 31
=================================================================
Total params: 11,241
Trainable params: 11,241
Non-trainable params: 0
_________________________________
About plotting the model, there is a couple of option that can be used while you plot your keras model. Here is one example:
keras.utils.plot_model(model, show_dtype=True,
show_layer_names=True, show_shapes=True,
to_file='model.png')
Can I define what layers I want to be define as "top".
As I understand, include_top= false will remove all denses layer on top.
I want to use "include_top=False" for transfert learning with my own models and don't want all my last dense layer to be automaticly remove.
to access the last dense layers you have to set include_top = True. In this way, you can create a sub-model with all the intermediate layers you want. here an example with VGG16
from tensorflow.keras.layers import *
from tensorflow.keras.models import *
from tensorflow.keras.applications import VGG16
vgg = VGG16(include_top = True, input_shape = (224,224,3))
new_layer = Dense(32)(vgg.layers[-3].output) # add new layer
sub_model = Model(vgg.input, new_layer)
the last layers of sub_model in this case are:
block5_conv3 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
block5_pool (MaxPooling2D) (None, 7, 7, 512) 0
_________________________________________________________________
flatten (Flatten) (None, 25088) 0
_________________________________________________________________
fc1 (Dense) (None, 4096) 102764544
_________________________________________________________________
dense_6 (Dense) (None, 32) 131104
I am new to Keras and deep learning and was working with MNIST on Keras. When I created a model using
model = models.Sequential()
model.add(layers.Dense(512,activation = 'relu',input_shape=(28*28,)))
model.add(layers.Dense(32,activation ='relu'))
model.add(layers.Dense(10,activation='softmax'))
and then I printed it
print(model)
output is
<keras.engine.sequential.Sequential at 0x7f3d554f6710>
My question is that is there any way to see a better result of Keras, meaning if i print model i can see that i have 3 hidden layers with first hidden layer having 512 hidden units and 784 input units, 2nd hidden layer having 512 input units and 32 hidden units and so on.
You can also try plot_model()
model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(512,activation = 'relu',input_shape=(28*28,)))
model.add(tf.keras.layers.Dense(32,activation ='relu'))
model.add(tf.keras.layers.Dense(10,activation='softmax'))
model.summary()
from keras.utils.vis_utils import plot_model
plot_model(model, show_shapes=True, show_layer_names=True)
model.summary() will print he entire model for you.
model = Sequential()
model.add(Dense(512,activation = 'relu',input_shape=(28*28,)))
model.add(Dense(32,activation ='relu'))
model.add(Dense(10,activation='softmax'))
model.summary()
Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense (Dense) (None, 512) 401920
_________________________________________________________________
dense_1 (Dense) (None, 32) 16416
_________________________________________________________________
dense_2 (Dense) (None, 10) 330
=================================================================
Total params: 418,666
Trainable params: 418,666
Non-trainable params: 0
____________________________
I am trying to build a 1D CNN but I can't get the right dimensions passed to my last dense layer
The architecture of my model is
model_CNN=Sequential()
model_CNN.add(Conv1D(14, 29, activation='relu', input_shape=(X_train.shape[1], 1)))
model_CNN.add(Conv1D(30, 22, activation='relu'))
model_CNN.add(Flatten())
model_CNN.add(Dense(176,activation='relu'))
model_CNN.add(Dense(Y_train.shape[1],activation='linear'))
With a summary that looks like
Layer (type) Output Shape Param #
=================================================================
conv1d_71 (Conv1D) (None, 3304, 14) 420
_________________________________________________________________
conv1d_72 (Conv1D) (None, 3283, 30) 9270
_________________________________________________________________
flatten_18 (Flatten) (None, 98490) 0
_________________________________________________________________
dense_102 (Dense) (None, 176) 17334416
_________________________________________________________________
dense_103 (Dense) (None, 5) 885
=================================================================
Total params: 17,344,991
Trainable params: 17,344,991
Non-trainable params: 0
When I try to fit my model, I confirm that my input shape is correct (240, 3332, 1), but then I get the following error
ValueError: Error when checking target: expected dense_103
to have 2 dimensions, but got array with shape (240, 5, 1)
So my flatten function is not creating a 1D array, but also somehow the input only fails on the second dense layer, not the first. What's going on?
I am using keras 1.1.1 in windows 7 with tensorflow backend.
I am trying to prepend the stock Resnet50 pretained model with an image downsampler. Below is my code.
from keras.applications.resnet50 import ResNet50
import keras.layers
# this could also be the output a different Keras model or layer
input = keras.layers.Input(shape=(400, 400, 1)) # this assumes K.image_dim_ordering() == 'tf'
x1 = keras.layers.AveragePooling2D(pool_size=(2,2))(input)
x2 = keras.layers.Flatten()(x1)
x3 = keras.layers.RepeatVector(3)(x2)
x4 = keras.layers.Reshape((200, 200, 3))(x3)
x5 = keras.layers.ZeroPadding2D(padding=(12,12))(x4)
m = keras.models.Model(input, x5)
model = ResNet50(input_tensor=m.output, weights='imagenet', include_top=False)
but I get an error which I am unsure how to fix.
builtins.Exception: Graph disconnected: cannot obtain value for tensor
Output("input_2:0", shape=(?, 400, 400, 1), dtype=float32) at layer
"input_2". The following previous layers were accessed without issue:
[]
You can use both the Functional API and Sequential approaches to solve this. See working example for both approaches below:
from keras.applications.ResNet50 import ResNet50
from keras.models import Sequential, Model
from keras.layers import AveragePooling2D, Flatten, RepeatVector, Reshape, ZeroPadding2D, Input, Dense
pretrained = ResNet50(input_shape=(224, 224, 3), weights='imagenet', include_top=False)
# Sequential method
model_1 = Sequential()
model_1.add(AveragePooling2D(pool_size=(2,2),input_shape=(400, 400, 1)))
model_1.add(Flatten())
model_1.add(RepeatVector(3))
model_1.add(Reshape((200, 200, 3)))
model_1.add(ZeroPadding2D(padding=(12,12)))
model_1.add(pretrained)
model_1.add(Dense(1))
# functional API method
input = Input(shape=(400, 400, 1))
x = AveragePooling2D(pool_size=(2,2),input_shape=(400, 400, 1))(input)
x = Flatten()(x)
x = RepeatVector(3)(x)
x = Reshape((200, 200, 3))(x)
x = ZeroPadding2D(padding=(12,12))(x)
x = pretrained(x)
preds = Dense(1)(x)
model_2 = Model(input,preds)
model_1.summary()
model_2.summary()
The summaries (replace resnet for xception):
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
average_pooling2d_1 (Average (None, 200, 200, 1) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 40000) 0
_________________________________________________________________
repeat_vector_1 (RepeatVecto (None, 3, 40000) 0
_________________________________________________________________
reshape_1 (Reshape) (None, 200, 200, 3) 0
_________________________________________________________________
zero_padding2d_1 (ZeroPaddin (None, 224, 224, 3) 0
_________________________________________________________________
xception (Model) (None, 7, 7, 2048) 20861480
_________________________________________________________________
dense_1 (Dense) (None, 7, 7, 1) 2049
=================================================================
Total params: 20,863,529
Trainable params: 20,809,001
Non-trainable params: 54,528
_________________________________________________________________
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_2 (InputLayer) (None, 400, 400, 1) 0
_________________________________________________________________
average_pooling2d_2 (Average (None, 200, 200, 1) 0
_________________________________________________________________
flatten_2 (Flatten) (None, 40000) 0
_________________________________________________________________
repeat_vector_2 (RepeatVecto (None, 3, 40000) 0
_________________________________________________________________
reshape_2 (Reshape) (None, 200, 200, 3) 0
_________________________________________________________________
zero_padding2d_2 (ZeroPaddin (None, 224, 224, 3) 0
_________________________________________________________________
xception (Model) (None, 7, 7, 2048) 20861480
_________________________________________________________________
dense_2 (Dense) (None, 7, 7, 1) 2049
=================================================================
Total params: 20,863,529
Trainable params: 20,809,001
Non-trainable params: 54,528
_________________________________________________________________
Both approaches work fine. If you plan on freezing the pretrained model and letting pre/post layers learn -- and afterward finetuning the model, the approach I found to work goes like so:
# given the same resnet model as before...
model = load_model('modelname.h5')
# pull out the nested model
nested_model = model.layers[5] # assuming the model is the 5th layer
# loop over the nested model to allow training
for l in nested_model.layers:
l.trainable=True
# insert the trainable pretrained model back into the original
model.layer[5] = nested_model