Train Multi-Input Keras NN with batch of training data - keras

Question

I would like to use Keras to train a multi-input NN with a batch of training data, but I'm not able to pass a set of input and output samples to execute a fit or a train_on_batch on the model.
My NN is defined as following:
i1 = keras.layers.Input(shape=(2,))
i2 = keras.layers.Input(shape=(2,))
i3 = keras.layers.Input(shape=(2,))
i_layer = keras.layers.Dense(2, activation='sigmoid')
embedded_i1 = i_layer(i1)
embedded_i2 = i_layer(i2)
embedded_i3 = i_layer(i3)
middle_concatenation = keras.layers.concatenate([embedded_i1, embedded_i2, embedded_i3], axis=1)
out = keras.layers.Dense(1, activation='sigmoid')(middle_concatenation)
model = keras.models.Model(inputs=[i1, i2, i3], outputs=out)
model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy'])
For example, an instance of the input (successfully used for predict the output) is the following:
[array([[0.1, 0.2]]), array([[0.3, 0.5]]), array([[0.1, 0.3]])]
But when I try to train my model with:
inputs = [[np.array([[0.1, 0.2]]), np.array([[0.3, 0.5]]), np.array([[0.1, 0.3]])],
[np.array([[0.2, 0.1]]), np.array([[0.5, 0.3]]), np.array([[0.3, 0.1]])]
]
outputs = np.ones(len(inputs))
model.fit(inputs, outputs)
I get this error:
ValueError: Error when checking model input: you are passing a list as input to your model, but the model expects a list of 3 Numpy arrays instead. The list you passed was: [[array([[ 0.1, 0.2]]), array([[ 0.3, 0.5]]), array([[ 0.1, 0.3]])], [array([[ 0.2, 0.1]]), array([[ 0.5, 0.3]]), array([[ 0.3, 0.1]])]]
What am I doing wrong?
How can I train a multi-input NN with a batch of input/output samples?
Thank you!

Answer 1

the problem is just incorrect formatting. You can't pass a list to keras, only numpy arrays, so when you have your data structured like
inputs = [[np.array([[0.1, 0.2]]), np.array([[0.3, 0.5]]), np.array([[0.1, 0.3]])],
[np.array([[0.2, 0.1]]), np.array([[0.5, 0.3]]), np.array([[0.3, 0.1]])]
]
You need to pass one list element into your model at a time. You will also need to pass one output value to the model at a time. To do this, structure you outputs like this
outputs = [np.ones(1) for x in inputs]
[array([ 1.]), array([ 1.])]
Then you can loop over the the fit function like this
for z in range(0,len(inputs)):
model.fit(inputs[z],outputs[z],batch_size=1)
you can also replace model.fit with model.train_on_batch() instead, see docs
however to avoid the loop, you could just have 3 numpy arrays stored in your inputs list and have you single outputs as a numpy array. If you only want to train on a single batch at a time, you could set your batch size to do that.
inputs = [np.array([[0.1, 0.2],[0.2, 0.1]]), np.array([[0.3, 0.5],[0.5, 0.3]]), np.array([[0.1, 0.3],[0.3, 0.1]])]
outputs = np.ones(inputs[0].shape[0])
model.fit(inputs,outputs,batch_size=1)

Answer 2

The problem is that right now you are using a list of lists as input, although keras expects a list of arrays.
You need to convert your list so that it looks like [array_inputs_1, array_inputs_2, array_inputs_3], where each input array is the array of inputs you would pass the model if it had only that input layer, you just put the 3 of them inside a list.
Using your data the correct input should be:
[np.array([[0.1, 0.2], [0.2, 0.1]]),
np.array([[0.3, 0.5], [0.5, 0.3]]),
np.array([[0.1, 0.3], [0.1, 0.3]])]
This way, as long as all 3 input arrays have the same number of elements, keras will know how to divide tham into batches.