Why is there extra paramaters in my simple CNN? - pytorch

Question

Here is my problem : when I look for the number of parameters in my first block, I see 36928 parameters (which is what I expect). But when I used this block to construct a model in an other class nn.Module, there are 1792 extra parameters and I can't figure out where they come from.
I put some code below to illustrate.
class Conv2dBlock(torch.nn.Module):
def __init__(self, in_filters, out_filters, kernel_size=3):
super(Conv2dBlock, self).__init__()
self.conv2d_seq = torch.nn.Sequential()
for k in range(2):
self.conv2d_seq.append(torch.nn.Conv2d(in_channels=in_filters, out_channels=out_filters, kernel_size=kernel_size, padding='same'))
self.conv2d_seq.append(torch.nn.ReLU())
in_filters = out_filters
def forward(self, input):
out = self.conv2d_seq(input)
return out
En then, I use this block in an other nn.Module :
class EncoderBlock(torch.nn.Module):
def __init__(self):
super(EncoderBlock, self).__init__()
self.conv2d = Conv2dBlock(3, 64)
self.maxpool = torch.nn.MaxPool2d(kernel_size=2)
def forward(self, input):
x = self.conv2d(input)
p = self.maxpool(x)
out = torch.nn.functional.dropout(p, 0.3)
return x, out
And finaly :
class UNet_model(torch.nn.Module):
def __init__(self):
super(UNet_model, self).__init__()
self.encoder_block1 = EncoderBlock()
def forward(self, input):
p1 = self.encoder_block1(input)
# I removed useless code
return p1
model = UNet_model()
summary(model, (3,128,128))
This last class constructs a model with 38 720 parameters, instead of 36 928. It seems there is an extra convolutional layer ((3,64, (3,3)) = 1792 params) applied twice to the input... I don't understand.
Can somebody take a look ?
Thanks !

Answer 1

First of all, torch.nn.Sequential() doesn't support append method, it should be changed to add_module, like this:
for k in range(2):
self.conv2d_seq.add_module(f"conv_{k}",torch.nn.Conv2d(in_channels=in_filters, out_channels=out_filters, kernel_size=kernel_size, padding='same'))
self.conv2d_seq.add_module(f"relu_{k}",torch.nn.ReLU())
in_filters = out_filters
Second, if you run torchinfo summary on the initial block you will see:
==========================================================================================
Layer (type:depth-idx) Output Shape Param #
==========================================================================================
Conv2dBlock [1, 64, 64, 64] --
├─Sequential: 1-1 [1, 64, 64, 64] --
│ └─Conv2d: 2-1 [1, 64, 64, 64] 1,792
│ └─ReLU: 2-2 [1, 64, 64, 64] --
│ └─Conv2d: 2-3 [1, 64, 64, 64] 36,928
│ └─ReLU: 2-4 [1, 64, 64, 64] --
==========================================================================================
Total params: 38,720
Trainable params: 38,720
Non-trainable params: 0
Total mult-adds (M): 158.60
==========================================================================================
Input size (MB): 0.05
Forward/backward pass size (MB): 4.19
Params size (MB): 0.15
Estimated Total Size (MB): 4.40
==========================================================================================
So you can see that you have two conv layers (1,792 + 36,928) as you specified 2 layers in your for loop: for k in range(2).