Custom Loss Function in TF2.0 - python-3.x

For a image segmentation problem, I need to write a custom loss function. I am getting below said error.
Code Base: https://www.tensorflow.org/tutorials/images/segmentation
Last layer:
Conv2DTrans (128,128,2) [Note that in my case it is only 2 values]
def call(self, y_true, y_pred):
ytrue = ytrue.numpy()
.....
Error:
AttributeError: 'Tensor' object has no attribute 'numpy'
I tried py_function and numpy_function but both return the same error
and also
with tf.compat.v1.Session() as sess:
for i,j in enumerate(sess.run(y_true),sess.run(y_pred)):
Current Model Layers:
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_82 (InputLayer) [(None, 128, 128, 3) 0
__________________________________________________________________________________________________
model_80 (Model) [(None, 64, 64, 96), 1841984 input_82[0][0]
__________________________________________________________________________________________________
sequential_160 (Sequential) (None, 8, 8, 512) 1476608 model_80[1][4]
__________________________________________________________________________________________________
concatenate_160 (Concatenate) (None, 8, 8, 1088) 0 sequential_160[0][0]
model_80[1][3]
__________________________________________________________________________________________________
sequential_161 (Sequential) (None, 16, 16, 256) 2507776 concatenate_160[0][0]
__________________________________________________________________________________________________
concatenate_161 (Concatenate) (None, 16, 16, 448) 0 sequential_161[0][0]
model_80[1][2]
__________________________________________________________________________________________________
sequential_162 (Sequential) (None, 32, 32, 128) 516608 concatenate_161[0][0]
__________________________________________________________________________________________________
concatenate_162 (Concatenate) (None, 32, 32, 272) 0 sequential_162[0][0]
model_80[1][1]
__________________________________________________________________________________________________
sequential_163 (Sequential) (None, 64, 64, 64) 156928 concatenate_162[0][0]
__________________________________________________________________________________________________
concatenate_163 (Concatenate) (None, 64, 64, 160) 0 sequential_163[0][0]
model_80[1][0]
__________________________________________________________________________________________________
conv2d_transpose_204 (Conv2DTra (None, 128, 128, 2) 2882 concatenate_163[0][0]
==================================================================================================
I need a numpy array to focus only more on 1's and not on zeroes. Now the metric and accuracy are overwhelmed by the presence of lot of zeros.
def tumor_loss(y_true,y_pred):
y_true = y_true.reshape((SHAPE,SHAPE))
y_pred = y_pred.reshape((SHAPE,SHAPE))
y_true_ind = np.where(y_true ==1)[1]
y_pred_ind = np.where(y_pred==1)[1]
if np.array_equal(y_true_ind,y_pred_ind):
return 0
if y_true_ind.shape[0] > y_pred_ind.shape[0]:
return y_true_ind.shape[0] - np.setdiff1d(y_true_ind,y_pred_ind).shape[0]
else:
return y_true_ind.shape[0] - np.setdiff1d(y_pred_ind,y_true_ind).shape[0]

If you are running on tf version >= 2.0, try using
model.compile(loss=custom_loss, optimizer='adam', run_eagerly=True)
if you are using Keras api.

Related

Number of feature maps do not increase after each block of 3D Resnet

I am trying to build 3D Resnet for small 3D patches of size [32,32,44] with one channel. In 2D Resnet, after each residual block, the size of images should reduce to half and the number of feature maps doubles as shown below
# function for creating an identity or projection residual module
def residual_module(layer_in, n_filters):
merge_input = layer_in
# check if the number of filters needs to be increase, assumes channels last format
if layer_in.shape[-1] != n_filters:
merge_input = Conv2D(n_filters, (1,1), padding='same', activation='relu', kernel_initializer='he_normal')(layer_in)
# conv1
conv1 = Conv2D(n_filters, (3,3), padding='same', activation='relu', kernel_initializer='he_normal')(layer_in)
# conv2
conv2 = Conv2D(n_filters, (3,3), padding='same', activation='linear', kernel_initializer='he_normal')(conv1)
# add filters, assumes filters/channels last
layer_out = add([conv2, merge_input])
# activation function
layer_out = Activation('relu')(layer_out)
return layer_out
# define model input
visible = Input(shape=(256, 256, 1))
layer = residual_module(visible,64)
layer_1 = residual_module(layer,128)
# create model
model = Model(inputs=visible, outputs=layer_1)
# summarize model
model.summary()
Result:
Model: "model_44"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_68 (InputLayer) [(None, 256, 256, 1) 0
__________________________________________________________________________________________________
conv2d_40 (Conv2D) (None, 256, 256, 64) 640 input_68[0][0]
__________________________________________________________________________________________________
conv2d_41 (Conv2D) (None, 256, 256, 64) 36928 conv2d_40[0][0]
__________________________________________________________________________________________________
conv2d_39 (Conv2D) (None, 256, 256, 64) 128 input_68[0][0]
__________________________________________________________________________________________________
add_207 (Add) (None, 256, 256, 64) 0 conv2d_41[0][0]
conv2d_39[0][0]
__________________________________________________________________________________________________
activation_52 (Activation) (None, 256, 256, 64) 0 add_207[0][0]
__________________________________________________________________________________________________
conv2d_43 (Conv2D) (None, 256, 256, 128 73856 activation_52[0][0]
__________________________________________________________________________________________________
conv2d_44 (Conv2D) (None, 256, 256, 128 147584 conv2d_43[0][0]
__________________________________________________________________________________________________
conv2d_42 (Conv2D) (None, 256, 256, 128 8320 activation_52[0][0]
__________________________________________________________________________________________________
add_208 (Add) (None, 256, 256, 128 0 conv2d_44[0][0]
conv2d_42[0][0]
__________________________________________________________________________________________________
activation_53 (Activation) (None, 256, 256, 128 0 add_208[0][0]
==================================================================================================
Total params: 267,456
Trainable params: 267,456
Non-trainable params: 0
However, adapting this code for 3D Resnet does not double the number of feature maps. As it can be seen in the below example that after first residual block channel dimension is still 1 and changes to 3 in the second block
def residual_module(layer_in, n_filters):
merge_input = layer_in
# check if the number of filters needs to be increase, assumes channels last format
if layer_in.shape[-1] != n_filters:
merge_input = Conv3D(n_filters, (1,1,1), padding='same', activation='relu', kernel_initializer='he_normal')(layer_in)
# conv1
conv1 = Conv3D(n_filters, (3,3,3), padding='same', activation='relu', kernel_initializer='he_normal')(layer_in)
# conv2
conv2 = Conv3D(n_filters, (3,3,3), padding='same', activation='linear', kernel_initializer='he_normal')(conv1)
# add filters, assumes filters/channels last
layer_out = add([conv2, merge_input])
# activation function
layer_out = Activation('relu')(layer_out)
return layer_out
# define model input
visible = Input(shape=(32,32,32,1))
layer = residual_module(visible,16)
layer_1 = residual_module(layer,32)
# create model
model = Model(inputs=visible, outputs=layer_1)
# summarize model
model.summary()
Result:
Model: "model_45"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_69 (InputLayer) [(None, 32, 32, 32, 0
__________________________________________________________________________________________________
conv3d_519 (Conv3D) (None, 32, 32, 32, 1 448 input_69[0][0]
__________________________________________________________________________________________________
conv3d_520 (Conv3D) (None, 32, 32, 32, 1 6928 conv3d_519[0][0]
__________________________________________________________________________________________________
conv3d_518 (Conv3D) (None, 32, 32, 32, 1 32 input_69[0][0]
__________________________________________________________________________________________________
add_209 (Add) (None, 32, 32, 32, 1 0 conv3d_520[0][0]
conv3d_518[0][0]
__________________________________________________________________________________________________
activation_54 (Activation) (None, 32, 32, 32, 1 0 add_209[0][0]
__________________________________________________________________________________________________
conv3d_522 (Conv3D) (None, 32, 32, 32, 3 13856 activation_54[0][0]
__________________________________________________________________________________________________
conv3d_523 (Conv3D) (None, 32, 32, 32, 3 27680 conv3d_522[0][0]
__________________________________________________________________________________________________
conv3d_521 (Conv3D) (None, 32, 32, 32, 3 544 activation_54[0][0]
__________________________________________________________________________________________________
add_210 (Add) (None, 32, 32, 32, 3 0 conv3d_523[0][0]
conv3d_521[0][0]
__________________________________________________________________________________________________
activation_55 (Activation) (None, 32, 32, 32, 3 0 add_210[0][0]
==================================================================================================
Total params: 49,488
Trainable params: 49,488
Non-trainable params: 0
_______________________________________________________________________________________________
What am I missing here?
Well, I figured out that the code is okay, except that the lines in model.summary() were truncated, so in fact feature maps in the first residual block are 16 and in the second they are 32. Increasing line_width was the catch
model.summary(line_length=110)
Results:
Model: "model_8"
______________________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==============================================================================================================
input_9 (InputLayer) [(None, 32, 32, 32, 1)] 0
______________________________________________________________________________________________________________
conv3d_136 (Conv3D) (None, 32, 32, 32, 16) 448 input_9[0][0]
______________________________________________________________________________________________________________
conv3d_137 (Conv3D) (None, 32, 32, 32, 16) 6928 conv3d_136[0][0]
______________________________________________________________________________________________________________
conv3d_135 (Conv3D) (None, 32, 32, 32, 16) 32 input_9[0][0]
______________________________________________________________________________________________________________
add_57 (Add) (None, 32, 32, 32, 16) 0 conv3d_137[0][0]
conv3d_135[0][0]
______________________________________________________________________________________________________________
activation_113 (Activation) (None, 32, 32, 32, 16) 0 add_57[0][0]
______________________________________________________________________________________________________________
conv3d_139 (Conv3D) (None, 32, 32, 32, 32) 13856 activation_113[0][0]
______________________________________________________________________________________________________________
conv3d_140 (Conv3D) (None, 32, 32, 32, 32) 27680 conv3d_139[0][0]
______________________________________________________________________________________________________________
conv3d_138 (Conv3D) (None, 32, 32, 32, 32) 544 activation_113[0][0]
______________________________________________________________________________________________________________
add_58 (Add) (None, 32, 32, 32, 32) 0 conv3d_140[0][0]
conv3d_138[0][0]
______________________________________________________________________________________________________________
activation_114 (Activation) (None, 32, 32, 32, 32) 0 add_58[0][0]
==============================================================================================================
Total params: 49,488
Trainable params: 49,488
Non-trainable params: 0

Keras multiple output expected shape and got shape

I am try to train a model which detect 128d vector to recognize face. Input of model is an image and output is 128d vector (regression) which get from "face_recognition" library.
When I put 128 output to train I got this error:
ValueError: Error when checking target: expected dense_24 to have shape (1,) but got array with shape (128,)
But when I try only one output, fit function works.
The strange part of that prediction shape is (1, 128) but I can't give 128 output to train.
Here is my model:
from keras.applications.vgg16 import VGG16
from keras.layers import Flatten, Dense
import keras
def build_facereg_disc():
# load model
model = VGG16(include_top=False, input_shape=(64, 64, 3))
# add new classifier layers
flat1 = Flatten()(model.outputs)
class1 = Dense(2048, activation='relu')(flat1)
output = Dense(128, activation='relu')(class1)
# define new model
model = models.Model(inputs=model.inputs, outputs=output)
# summarize
return model
facereg_disc = build_facereg_disc()
facereg_disc.compile(optimizer=keras.optimizers.Adam(), # Optimizer
# Loss function to minimize
loss=keras.losses.SparseCategoricalCrossentropy(),
# List of metrics to monitor
metrics=['binary_crossentropy'])
And summary:
Model: "model_27"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_20 (InputLayer) (None, 64, 64, 3) 0
_________________________________________________________________
block1_conv1 (Conv2D) (None, 64, 64, 64) 1792
_________________________________________________________________
block1_conv2 (Conv2D) (None, 64, 64, 64) 36928
_________________________________________________________________
block1_pool (MaxPooling2D) (None, 32, 32, 64) 0
_________________________________________________________________
block2_conv1 (Conv2D) (None, 32, 32, 128) 73856
_________________________________________________________________
block2_conv2 (Conv2D) (None, 32, 32, 128) 147584
_________________________________________________________________
block2_pool (MaxPooling2D) (None, 16, 16, 128) 0
_________________________________________________________________
block3_conv1 (Conv2D) (None, 16, 16, 256) 295168
_________________________________________________________________
block3_conv2 (Conv2D) (None, 16, 16, 256) 590080
_________________________________________________________________
block3_conv3 (Conv2D) (None, 16, 16, 256) 590080
_________________________________________________________________
block3_pool (MaxPooling2D) (None, 8, 8, 256) 0
_________________________________________________________________
block4_conv1 (Conv2D) (None, 8, 8, 512) 1180160
_________________________________________________________________
block4_conv2 (Conv2D) (None, 8, 8, 512) 2359808
_________________________________________________________________
block4_conv3 (Conv2D) (None, 8, 8, 512) 2359808
_________________________________________________________________
block4_pool (MaxPooling2D) (None, 4, 4, 512) 0
_________________________________________________________________
block5_conv1 (Conv2D) (None, 4, 4, 512) 2359808
_________________________________________________________________
block5_conv2 (Conv2D) (None, 4, 4, 512) 2359808
_________________________________________________________________
block5_conv3 (Conv2D) (None, 4, 4, 512) 2359808
_________________________________________________________________
block5_pool (MaxPooling2D) (None, 2, 2, 512) 0
_________________________________________________________________
flatten_10 (Flatten) (None, 2048) 0
_________________________________________________________________
dense_23 (Dense) (None, 2048) 4196352
_________________________________________________________________
dense_24 (Dense) (None, 128) 262272
=================================================================
Total params: 19,173,312
Trainable params: 19,173,312
Non-trainable params: 0
Here is preprocessing:
dir_data = "data_faces/img_align_celeba/"
Ntrain = 2000
Ntest = 100
nm_imgs = np.sort(os.listdir(dir_data))
## name of the jpg files for training set
nm_imgs_train = nm_imgs[:Ntrain]
## name of the jpg files for the testing data
nm_imgs_test = nm_imgs[Ntrain:Ntrain + Ntest]
img_shape = (64, 64, 3)
def get_npdata(nm_imgs_train):
X_train = []
for i, myid in enumerate(nm_imgs_train):
image = load_img(dir_data + "/" + myid,
target_size=img_shape[:2])
image = img_to_array(image)/255.0
X_train.append(image)
X_train = np.array(X_train)
return(X_train)
X_train = get_npdata(nm_imgs_train)
X_train.shape = (2000, 64, 64, 3)
y_train.shape = (2000, 128)
I use batch size like:
idx = np.random.randint(0, X_train.shape[0], half_batch)
imgs = X_train[idx]
labels = y_train[idx]
reg_d_loss_real = facereg_disc.train_on_batch(imgs, labels)
Your issue comes from your loss function. As explained in the doc, SparseCategoricalCrossentropy expects each sample in y_true to be an integer encoding the class, whereas CategoricalCrossentropy expects a one-hot encoded representation (which is your case).
So, switch to CategoricalCrossentropy and you should be fine.
However, to reproduce, I had to change:
flat1 = Flatten()(model.outputs)
To:
flat1 = Flatten()(model.outputs[0])

Can not convert Resnet model trained with Keras to CoreML model

I need to convert a resnet50 model to CoreML model.
The Keras model trained is working correctly. I tried to convert it to Coreml but here is the error I get using coremltools :
ValueError: Keras layer '<class 'keras.layers.core.Lambda'>' not supported.
It seems that I have lambda functions in my model and Coreml does not support it... but what I do not understand where those lambda functions are coming from as I just used a standard resnet50 network for transfer learning. I only changed the last 1000-dense layer to a 4-dense layer, here is my code:
from keras.applications.resnet50 import ResNet50, preprocess_input
full_imagenet_model = ResNet50(weights='imagenet')
output = full_imagenet_model.layers[-2].output
base_model = Model(full_imagenet_model.input, output)
top_model = Sequential()
top_model.add(Dense(4, input_dim=2048, activation='softmax'))
top_model.compile(optimizer=Adam(lr=1e-4),
loss='categorical_crossentropy', metrics=['accuracy'])
model = Model(base_model.input, top_model(base_model.output))
Here is the beginning and end of model summary:
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) (None, 224, 224, 3) 0
__________________________________________________________________________________________________
conv1_pad (ZeroPadding2D) (None, 230, 230, 3) 0 input_1[0][0]
__________________________________________________________________________________________________
conv1 (Conv2D) (None, 112, 112, 64) 9472 conv1_pad[0][0]
__________________________________________________________________________________________________
bn_conv1 (BatchNormalization) (None, 112, 112, 64) 256 conv1[0][0]
__________________________________________________________________________________________________
activation_1 (Activation) (None, 112, 112, 64) 0 bn_conv1[0][0]
__________________________________________________________________________________________________
pool1_pad (ZeroPadding2D) (None, 114, 114, 64) 0 activation_1[0][0]
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D) (None, 56, 56, 64) 0 pool1_pad[0][0]
__________________________________________________________________________________________________
res2a_branch2a (Conv2D) (None, 56, 56, 64) 4160 max_pooling2d_1[0][0]
________________________________________________________________________________________________
(...)
__________________________________________________________________________________
add_16 (Add) (None, 7, 7, 2048) 0 bn5c_branch2c[0][0]
activation_46[0][0]
__________________________________________________________________________________________________
activation_49 (Activation) (None, 7, 7, 2048) 0 add_16[0][0]
__________________________________________________________________________________________________
avg_pool (GlobalAveragePooling2 (None, 2048) 0 activation_49[0][0]
__________________________________________________________________________________________________
sequential_1 (Sequential) (None, 4) 8196 avg_pool[0][0]
==================================================================================================
Total params: 23,595,908
Trainable params: 23,542,788
Non-trainable params: 53,120
What is weird is when I load the model I trained and call the summary, here is what I get:
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_3 (InputLayer) (None, 224, 224, 3) 0
__________________________________________________________________________________________________
lambda_3 (Lambda) (None, 224, 224, 3) 0 input_3[0][0]
__________________________________________________________________________________________________
lambda_4 (Lambda) (None, 224, 224, 3) 0 input_3[0][0]
__________________________________________________________________________________________________
model_5 (Model) (None, 4) 23595908 lambda_3[0][0]
lambda_4[0][0]
__________________________________________________________________________________________________
sequential_2 (Concatenate) (None, 4) 0 model_5[1][0]
model_5[2][0]
==================================================================================================
Total params: 23,595,908
Trainable params: 23,542,788
Non-trainable params: 53,120
I have no idea where the lambda layers come from.. any idea?
For information, here is how the training is done:
opt = Adam(lr=1e-3)
parallel_model = multi_gpu_model(model, gpus=2)
parallel_model.compile(optimizer=opt, loss='categorical_crossentropy',
metrics=['accuracy'])
history = parallel_model.fit_generator(train_flow, train_flow.n // train_flow.batch_size,
epochs=200,
validation_data=val_flow,
validation_steps=val_flow.n,
callbacks=[clr, tensorboard, cb_checkpointer, cb_early_stopper])
Thanks for the help
Edit1
And here is how I save the model:
from tensorflow.python.keras.callbacks import EarlyStopping, ModelCheckpoint
from pyimagesearch.clr_callback import CyclicLR
cb_early_stopper = EarlyStopping(monitor = 'val_acc', patience = 30)
cb_checkpointer = ModelCheckpoint(filepath = 'SAVED_MODELS/EPOCH:50_DataAug:Yes_Monitor:val-acc_DB2.hdf5', monitor = 'val_acc', save_best_only = True, mode = 'auto')
tensorboard = TensorBoard(log_dir="logs/{}".format('model_EPOCH:50_DataAug:Yes_Monitor:val-acc_DB2'))
clr = CyclicLR(
mode=CLR_METHOD,
base_lr=MIN_LR,
max_lr=MAX_LR,
step_size= STEP_SIZE * (train_flow.n // train_flow.batch_size))
Ok, I think I found the issue. The code I posted missed a part about multi_gpu_model used for training.
The probleme is that using multi_gpu is creating lambdas functions in the summary to load data in parralel.
I am looking now to implement the lambda functions in coremltools.

Adding Conv Layer in front of pretrained model gives ValueError

I want to combine a pretrained VGG16 model with a special input block, which is an input layer and a convolutional layer. The goal is to use a pre-trained RGB VGG16 imagenet model on grayscale images:
from keras.applications.vgg16 import VGG16
from keras.layers.convolutional import Conv2D
from keras.layers import Input
from keras.models import Model
img_height = 299
img_width = 299
def input_block(img_height = 299, img_width = 299):
input_shape = (img_height, img_width, 1)
img_input = Input(shape=input_shape, name = 'grayscale_input_layer')
x = Conv2D(3, (3,3), padding= 'same', name = 'grayscale_RGB_layer')(img_input)
return x
pretrained_model = VGG16(weights = 'imagenet', include_top=False, input_tensor = input_block(img_height, img_width))
When I set the weight initalization of VGG16() to 'None', the model builds correctly, with the following desired structure:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
grayscale_input_layer (Input (None, 299, 299, 1) 0
_________________________________________________________________
grayscale_RGB_layer (Conv2D) (None, 299, 299, 3) 30
_________________________________________________________________
block1_conv1 (Conv2D) (None, 299, 299, 64) 1792
_________________________________________________________________
block1_conv2 (Conv2D) (None, 299, 299, 64) 36928
_________________________________________________________________
block1_pool (MaxPooling2D) (None, 149, 149, 64) 0
_________________________________________________________________
block2_conv1 (Conv2D) (None, 149, 149, 128) 73856
_________________________________________________________________
block2_conv2 (Conv2D) (None, 149, 149, 128) 147584
_________________________________________________________________
block2_pool (MaxPooling2D) (None, 74, 74, 128) 0
_________________________________________________________________
block3_conv1 (Conv2D) (None, 74, 74, 256) 295168
_________________________________________________________________
block3_conv2 (Conv2D) (None, 74, 74, 256) 590080
_________________________________________________________________
block3_conv3 (Conv2D) (None, 74, 74, 256) 590080
_________________________________________________________________
block3_pool (MaxPooling2D) (None, 37, 37, 256) 0
_________________________________________________________________
block4_conv1 (Conv2D) (None, 37, 37, 512) 1180160
_________________________________________________________________
block4_conv2 (Conv2D) (None, 37, 37, 512) 2359808
_________________________________________________________________
block4_conv3 (Conv2D) (None, 37, 37, 512) 2359808
_________________________________________________________________
block4_pool (MaxPooling2D) (None, 18, 18, 512) 0
_________________________________________________________________
block5_conv1 (Conv2D) (None, 18, 18, 512) 2359808
_________________________________________________________________
block5_conv2 (Conv2D) (None, 18, 18, 512) 2359808
_________________________________________________________________
block5_conv3 (Conv2D) (None, 18, 18, 512) 2359808
_________________________________________________________________
block5_pool (MaxPooling2D) (None, 9, 9, 512) 0
=================================================================
Total params: 14,714,718
Trainable params: 14,714,718
Non-trainable params: 0
_________________________________________________________________
None
However, when I set the weight initialization to 'imagenet',
I get the following error:
ValueError: You are trying to load a weight file containing 13 layers into a model with 14 layers.
This error makes sense, since I have added two layers in front of the VGG16 model instead of a single layer.
As a workaround, I have tried the following:
def input_block_model(img_height = 299, img_width = 299):
input_shape = (img_height, img_width, 1)
img_input = Input(shape=input_shape, name = 'grayscale_input_layer')
x = Conv2D(3, (3,3), padding= 'same', name = 'grayscale_RGB_layer')(img_input)
model = Model(img_input, x, name='input_block_model')
return model
input_model = input_block_model(299,299)
pretrained_model = VGG16(weights = "imagenet", include_top=False)
combined_model = Model(input_model.input,
pretrained_model(input_model.output))
print(combined_model.summary())
Then, the model structure is:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
grayscale_input_layer (Input (None, 299, 299, 1) 0
_________________________________________________________________
grayscale_RGB_layer (Conv2D) (None, 299, 299, 3) 30
_________________________________________________________________
vgg16 (Model) multiple 14714688
=================================================================
Total params: 14,714,718
Trainable params: 14,714,718
Non-trainable params: 0
_________________________________________________________________
None
The disadvantage of this structure, is that I cannot set properties of layers within the VGG16 model. I want to freeze certain layers for example in this model, which I cannot access via combined_model.layers. Does anyone have a working solution, such that I get the model structure as with the 'None' initialization, but with pretrained ImageNet weights?
You can freeze or train layers using combined_model.layers[2].layers as mentioned in the comment above. You can may be simplify the model as follows:
```
img_input = Input(shape=(img_height, img_width, 1), name = 'grayscale_input_layer')
x = Conv2D(3, (3,3), padding= 'same', name = 'grayscale_RGB_layer')(img_input)
x = VGG16(weights = None, include_top=False)(x)
model = Model(img_input, x)
model.summary()
for layer in model.layers[2].layers:
layer.trainable = False
```

how to visualize images after dense layer in cnn model in keras

My CNN model contains convolution layer and dense layers. I am able to visualize images and filter of convolution layers with help of below code, but unable to see output images after dense layers (only images, because there is no filters). When i tried using below code i am getting error:
File "<ipython-input-25-e8e4d4494672>", line 35, in <module>
num_of_featuremaps=feature_maps.shape[2]
IndexError: tuple index out of range
#and after that some blank space
code is following:
def get_featuremaps(model, layer_idx, X_batch):
get_activations = K.function([model.layers[0].input, K.learning_phase()],[model.layers[layer_idx].output,])
activations = get_activations([X_batch,0])
return activations
layer_num=11
filter_num=0
test_image=x[0]
test_image_show=test_image[:,:,0]
plt.axis('off')
test_image= np.expand_dims(test_image, axis=0)
print (test_image.shape)
activations = get_featuremaps(model, int(layer_num),test_image)
print (np.shape(activations))
feature_maps = activations[0][0]
print (np.shape(feature_maps))
if K.image_dim_ordering()=='th':
feature_maps=np.rollaxis((np.rollaxis(feature_maps,2,0)),2,0)
print (feature_maps.shape)
fig=plt.figure(figsize=(16,16))
#plt.imshow(feature_maps[:,:,filter_num],cmap='gray')
#plt.savefig("featuremaps-layer-{}".format(layer_num) + "-filternum-{}".format(filter_num)+'.jpg')
num_of_featuremaps=feature_maps.shape[2]
fig=plt.figure(figsize=(16,16))
plt.title("featuremaps-layer-{}".format(layer_num))
subplot_num=int(np.ceil(np.sqrt(num_of_featuremaps)))
for i in range(int(num_of_featuremaps)):
ax = fig.add_subplot(subplot_num, subplot_num, i+1)
ax.imshow(feature_maps[:,:,i],cmap='gray')
plt.xticks([])
plt.yticks([])
plt.tight_layout()
plt.show()
from mpl_toolkits.axes_grid1 import make_axes_locatable
def nice_imshow(ax, data, vmin=None, vmax=None, cmap=None):
"""Wrapper around pl.imshow"""
if cmap is None:
cmap = cm.jet
if vmin is None:
vmin = data.min()
if vmax is None:
vmax = data.max()
divider = make_axes_locatable(ax)
cax = divider.append_axes("right", size="5%", pad=0.05)
im = ax.imshow(data, vmin=vmin, vmax=vmax, interpolation='nearest', cmap=cmap)
pl.colorbar(im, cax=cax)
model looks like following:
Layer (type) Output Shape Param #
=================================================================
conv2d_37 (Conv2D) (None, 49, 49, 32) 160
_________________________________________________________________
conv2d_38 (Conv2D) (None, 48, 48, 32) 4128
_________________________________________________________________
max_pooling2d_19 (MaxPooling (None, 24, 24, 32) 0
_________________________________________________________________
dropout_28 (Dropout) (None, 24, 24, 32) 0
_________________________________________________________________
conv2d_39 (Conv2D) (None, 23, 23, 64) 8256
_________________________________________________________________
conv2d_40 (Conv2D) (None, 22, 22, 64) 16448
_________________________________________________________________
max_pooling2d_20 (MaxPooling (None, 11, 11, 64) 0
_________________________________________________________________
dropout_29 (Dropout) (None, 11, 11, 64) 0
_________________________________________________________________
flatten_10 (Flatten) (None, 7744) 0
_________________________________________________________________
dense_19 (Dense) (None, 256) 1982720
_________________________________________________________________
dropout_30 (Dropout) (None, 256) 0
_________________________________________________________________
dense_20 (Dense) (None, 2) 514
=================================================================
Total params: 2,012,226
Trainable params: 2,012,226
Non-trainable params: 0
_____________________________________
where pl is pylab and plt is matplotlib.

Resources