Theano MLP with 2 hidden layers throws Shape Mismatch error - theano

Question

I'm approaching to neural networks implementations, trying to build a working MLP using Theano. Following the tutorial, I tried to enhance the net by adding a layer, for a total of two hidden layers each with the same amount of units (250). The problem is that when I run the script I meet "Shape mismatch" ValueError. My code is a modified version of the tutorial code that can be found here http://deeplearning.net/tutorial/mlp.html.
The part I modified is the snippet-2, namely the MLP object, as follows:
class MLP(object):
def __init__(self, rng, input, n_in, n_hidden, n_out):
"""Initialize the parameters for the multilayer perceptron
:type rng: numpy.random.RandomState
:param rng: a random number generator used to initialize weights
:type input: theano.tensor.TensorType
:param input: symbolic variable that describes the input of the
architecture (one minibatch)
:type n_in: int
:param n_in: number of input units, the dimension of the space in
which the datapoints lie
:type n_hidden: int
:param n_hidden: number of hidden units
:type n_out: int
:param n_out: number of output units, the dimension of the space in
which the labels lie
"""
self.hiddenLayer1 = HiddenLayer(
rng=rng,
input=input,
n_in=n_in,
n_out=n_hidden,
activation=T.tanh
)
#try second hidden layer
self.hiddenLayer2 = HiddenLayer(
rng=rng,
input=self.hiddenLayer1.output,
n_in=n_in,
n_out=n_hidden,
activation=T.tanh
)
# The logistic regression layer gets as input the hidden units
# of the hidden layer
self.logRegressionLayer = LogisticRegression(
input=self.hiddenLayer2.output,
n_in=n_hidden,
n_out=n_out
)
# end-snippet-2 start-snippet-3
# L1 norm ; one regularization option is to enforce L1 norm to
# be small
self.L1 = (
abs(self.hiddenLayer1.W).sum()
+ abs(self.hiddenLayer2.W).sum()
+ abs(self.logRegressionLayer.W).sum()
)
# square of L2 norm ; one regularization option is to enforce
# square of L2 norm to be small
self.L2_sqr = (
(self.hiddenLayer1.W ** 2).sum()
+ (self.hiddenLayer2.W ** 2).sum()
+ (self.logRegressionLayer.W ** 2).sum()
)
# negative log likelihood of the MLP is given by the negative
# log likelihood of the output of the model, computed in the
# logistic regression layer
self.negative_log_likelihood = (
self.logRegressionLayer.negative_log_likelihood
)
# same holds for the function computing the number of errors
self.errors = self.logRegressionLayer.errors
# the parameters of the model are the parameters of the two layer it is
# made out of
self.params = self.hiddenLayer1.params + self.hiddenLayer2.params + self.logRegressionLayer.params
# end-snippet-3
# keep track of model input
self.input = input
I also removed some comments for readability. The output error I get is:
ValueError: Shape mismatch: x has 250 cols (and 20 rows) but y has 784
rows (and 250 cols) Apply node that caused the error:
Dot22(Elemwise{Composite{tanh((i0 + i1))}}[(0, 0)].0, W) Inputs types:
[TensorType(float64, matrix), TensorType(float64, matrix)] Inputs
shapes: [(20, 250), (784, 250)] Inputs strides: [(2000, 8), (2000, 8)]
Inputs values: ['not shown', 'not shown']

Answer 1

The size of the input to layer 2 needs to be the same size as the output from layer 1.
hiddenLayer2 takes hiddenLayer1 as input and hiddenLayer1.n_out == n_hidden but 'hiddenLayer2.n_in == n_in'. In this case n_hidden == 250 and n_in == 784. They should match but don't hence the error.
The solution is to make hiddenLayer2.n_in == hiddenLayer1.n_out.