Using an Embedding layer in the Keras functional API - python-3.x

Question

Doing basic things with the Keras Functional API seems to produce errors. For example, the following fails:
from keras.layers import InputLayer, Embedding
input = InputLayer(name="input", input_shape=(1, ))
embedding = Embedding(10000, 64)(input)
This produces the error:
AttributeError: 'str' object has no attribute 'base_dtype'
I can then "cheat" by using the input_length argument but this then fails when I try to concatenate two such embeddings:
from keras.layers import InputLayer, Embedding, Concatenate
embedding1 = Embedding(10000, 64, input_length=1)
embedding2 = Embedding(10000, 64, input_length=1)
concat = Concatenate()([embedding1 , embedding2])
This gives the error:
TypeError: 'NoneType' object is not subscriptable
Same error when I use "concatenate" (lower case) instead (some sources seem to say that this should be used instead if using the functional API).
What am I doing wrong?
I am on tensorflow version 2.3.1, keras version 2.4.3, python version 3.6.7

Answer 1

I strongly suggest to use tf.keras and not keras.
It doesn't work because InputLayer is an instance of keras.Layer, whereas keras.layers.Input is an instance of Tensor. The argument to layer.__call__() should be Tensor and not keras.Layer.
import tensorflow as tf
inputs = tf.keras.layers.Input((1,))
print(type(inputs)) # <class 'tensorflow.python.framework.ops.Tensor'>
input_layer = tf.keras.layers.InputLayer(input_shape=(1,))
print(type(input_layer)) # <class 'tensorflow.python.keras.engine.input_layer.InputLayer'>
You use InputLayer with Sequential API. When you use functional API you should use tf.keras.layers.Input() instead:
import tensorflow as tf
inputs = tf.keras.layers.Input((1, ), name="input", )
embedding = tf.keras.layers.Embedding(10000, 64)(inputs)
Same with the second example:
import tensorflow as tf
inputs = tf.keras.layers.Input((1, ), name="input", )
embedding1 = tf.keras.layers.Embedding(10000, 64)(inputs)
embedding2 = tf.keras.layers.Embedding(10000, 64)(inputs)
concat = tf.keras.layers.Concatenate()([embedding1, embedding2])