Restricting the output values of layers in Keras - keras

Question

I have defined my MLP in the code below. I want to extract the values of layer_2.
def gater(self):
dim_inputs_data = Input(shape=(self.train_dim[1],))
dim_svm_yhat = Input(shape=(3,))
layer_1 = Dense(20,
activation='sigmoid')(dim_inputs_data)
layer_2 = Dense(3, name='layer_op_2',
activation='sigmoid', use_bias=False)(layer_1)
layer_3 = Dot(1)([layer_2, dim_svm_yhat])
out_layer = Dense(1, activation='tanh')(layer_3)
model = Model(input=[dim_inputs_data, dim_svm_yhat], output=out_layer)
adam = optimizers.Adam(lr=0.01)
model.compile(loss='mse', optimizer=adam, metrics=['accuracy'])
return model
Suppose the output of layer_2 is below in matrix form
0.1 0.7 0.8
0.1 0.8 0.2
0.1 0.5 0.5
....
I would like below to be fed into layer_3 instead of above
0 0 1
0 1 0
0 1 0
Basically, I want the first maximum values to be converted to 1 and other to 0.
How can this be achieved in keras?.

Answer 1

Who decides the range of output values?
Output range of any layer in a neural network is decided by the activation function used for that layer. For example, if you use tanh as your activation function, your output values will be restricted to [-1,1] (and the values are continuous, check how the values get mapped from [-inf,+inf] (input on x-axis) to [-1,+1] (output on y-axis) here, understanding this step is very important)
What you should be doing is add a custom activation function that restricts your values to a step function i.e., either 1 or 0 for [-inf, +inf] and apply it to that layer.
How do I know which function to use?
You need to create y=some_function that satisfies all your needs (the input to output mapping) and convert that to Python code just like this one:
from keras import backend as K
def binaryActivationFromTanh(x, threshold) :
# convert [-inf,+inf] to [-1, 1]
# you can skip this step if your threshold is actually within [-inf, +inf]
activated_x = K.tanh(x)
binary_activated_x = activated_x > threshold
# cast the boolean array to float or int as necessary
# you shall also cast it to Keras default
# binary_activated_x = K.cast_to_floatx(binary_activated_x)
return binary_activated_x
After making your custom activation function, you can use it like
x = Input(shape=(1000,))
y = Dense(10, activation=binaryActivationFromTanh)(x)
Now test the values and see if you are getting the values like you expected. You can now throw this piece into a bigger neural network.
I strongly discourage adding new layers to add restriction to your outputs, unless it is solely for activation (like keras.layers.LeakyReLU).

Answer 2

Use Numpy in between. Here is an example with a random matrix:
a = np.random.random((5, 5)) # simulate random value output of your layer
result = (a == a.max(axis=1)[:,None]).astype(int)
See also this thread: Numpy: change max in each row to 1, all other numbers to 0
You than feed in result as input to your next layer.
For wrapping the Numpy calculation you could use the Lambda layer. See examples here: https://keras.io/layers/core/#lambda
Edit:
Suggestion doesn´t work. I keep answer only to keep related comments.