Keras how is the input layer defined (basic understanding) number of layers - keras

I try to follow a tutorial.
The building of the model starts with
netw <- keras_model_sequential()
### one input layer, one output layer and one hidden layer
netw %>% layer_dense(units = 500, activation = "relu", input_shape = c(6)) %>%
layer_dense(units=300, activation="relu") %>%
layer_dense(units=2, activation="softmax")
units define the number of nodes. I am confused about the number of layers. The input layer should have 6 nodes, since there are 6 features. Why does it have units=500? Does this really specify an input layer with 6 nodes and a second layer with 500 nodes? So there would be 4 layers instead of the three stated in the comment?

The input shape does refer to the size of the input layer. This is the number of data point being fed into the network while units = 500 is the number of network nodes the data is being passed to.
Here is a visualization of your model:
As you can see, there are indeed four layers in the model. The input layer is not generally counted because every network will generally contain an input layer. More specifically, the input layer is not a computation layer, so there are only three levels of computation being carried out.

Related

What do units and layers do in neural network?

model = keras.Sequential([
# the hidden ReLU layers
layers.Dense(units=4, activation='relu', input_shape=[2]),
layers.Dense(units=3, activation='relu'),
# the linear output layer
layers.Dense(units=1),
])
The above is a Keras sequential model example from Kaggle. I'm having a problem understanding these two things.
Are the units the number of nodes in a hidden layer? I see some people put 250 or what ever. What does the number do when it gets changed higher or lower?
Why would another hidden layer need to be added? What does it actually do the data to add more and more layers?
Answers in brief
units is representing how many neurons in a particular layer.When you have higher number,model has higher parameters to update during learning.Same thing goes to layers as well.(more layers and more neurons take more time to train the model).selecting how many neurons is depend on the use case and dataset and model architecture.
When you have more hidden layers, you have more parameters to update.More parameters and layers meaning model is able to understand complex relationships hidden in the data. For example when you have a image classification(multiple), you need more deep layers with neurons to understand the features in the image, which use to classify in final layer.
play with tensorflow playground,it will give great idea when you change the layers and neurons.

Keras LSTM network predicts all signals belong to the same category (among 3 different ones)

I'm working on a project that involves signal classification. I'm trying different models of ANN using keras to see which one is better, for now focusing in simple networks but I'm struggling with the LSTM one, following this example: https://machinelearningmastery.com/how-to-develop-rnn-models-for-human-activity-recognition-time-series-classification/.
My inputs are 1D signals that I get from an electronic sensor that will be divided in 3 different categories. See here one signal of two different categories so you see they are quite different over time.
https://www.dropbox.com/s/9ctdegtuyjamp48/example_signals.png?dl=0
To start with a simple model, we are trying the following simple model. Since signals are of different length, a masking process has been performed on them, enlarge each one to the longest one with the masked value of -1000 (impossible value to happen in our signal). Data is correctly reshaped from 2D to 3D (since is needed in 3D for the LSTM layer) using the following command since I only have a feature:
Inputs = Inputs.reshape((Inputs.shape[0],Inputs.shape[1],1))
Then, data is divided in training and validation one and feed into the following model:
model = Sequential()
model.add(Masking(mask_value=-1000, input_shape=(num_steps,1)))
model.add(LSTM(20, return_sequences=False))
model.add(Dense(15, activation='sigmoid'))
model.add(Dense(3, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['categorical_accuracy'])
However, for some reason, each time the network is trained, it always predicts that ALL signals are from the same category, possibly a different category each time is trained, usually the one with more cases inside the input data. If I force the network to be trained with same amount of data for each category, keeps giving me the same result.
I don't think this behaviour is normal: having bad accuracy could happen but this must have to do with some elementary error in the model that I'm not spotting since the given data in the training is correctly inputted, no errors there, rechecked multiple times. Anyone has any idea why this is happening?
Let me know if any more info could be useful to add to this post.
Just for any curious reader: at the end I could resolve it by normalizing the data.
def LSTM_input_normalize(inputs):
new_inputs = []
for in_ in inputs:
if -1000 in in_:
start_idx = np.where(in_ == -1000)[0][0] # index of the first "-1000" in the sequence
else:
start_idx = in_.shape[0]
# compute mean and std of the current sequence
curr_mean = np.mean(in_[:start_idx])
curr_std = np.std(in_[:start_idx])
# normalize the single sequence
in_[:start_idx] = (in_[:start_idx] - curr_mean) / curr_std
new_inputs.append(in_)
return np.array(new_inputs)

How should I change the model if accuracy is very low?

I am new to deep learning and I want to build an image classifier using CNN(keras). I have built a model with 2 convolution layers (filters = 32 , kernel = 3x3) followed by a MaxPooling layer(2x2) and this repeated 2 times. Finally 2 fully connected layers. I am getting an accuracy of 50%. My question is how do we choose the model to begin with. Like how do we decide that there should be 2 convolution layers followed by a MaxPooling layer or 1 convolution and 1 MaxPooling layer. Also how do we choose the number of filters in each convolution layer and the kernel size.
If my model is not working then how to decide what changes to be made to the model .
model = Sequential()
model.add(Convolution2D(32,3,3,input_shape=
(280,280,3),activation='relu'))
model.add(Convolution2D(32,3,3,activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
#model.add(Dropout(0.25))
model.add(Convolution2D(64,3,3,activation='relu'))
model.add(Convolution2D(64,3,3,activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
#model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(output_dim=256 , activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(output_dim=5,activation='softmax'))
I am getting an accuracy of 50% after 5 epochs. What changes should i make in my model?
Let us first start with the more straightforward part. Knowing the number of input and output layers and the number of their neurons is the easiest part. Every network has a single input layer and a single output layer. The number of neurons in the input layer equals the number of input variables in the data being processed. The number of neurons in the output layer equals the number of outputs associated with each input.
But the challenge is knowing the number of hidden layers and their neurons.
The answer is you cannot analytically calculate the number of layers or the number of nodes to use per layer in an artificial neural network to address a specific real-world predictive modeling problem.
The number of layers and the number of nodes in each layer are model hyperparameters that you must specify and learn.
You must discover the answer using a robust test harness and controlled experiments. Regardless of the heuristics, you might encounter, all answers will come back to the need for careful experimentation to see what works best for your specific dataset.
Again the filter size is one such hyperparameter you should specify before training your network.
For an image recognition problem, if you think that a big amount of pixels are necessary for the network to recognize the object you will use large filters (as 11x11 or 9x9). If you think what differentiates objects are some small and local features you should use small filters (3x3 or 5x5).
These are some tips but do not exist any rules.
There are many tricks to increase the accuracy of your deep learning model. Kindly refer to this link Improve deep learning model performance.
Hope this will help you.

Why do we need to include_top=False if we need to change the input_shape?

As far as I know, the input tuple enters from the convolution blocks.
So if we want to change the input_tuple shape, modifying convolutions would make sense.
Why do we need to include_top=False and remove the fully connected layers at the end?
On the other hand, if we have different number of classes,Keras has an option to change the softmax layer using no_of_classes
I know that I am the one missing something here. Please help me
Example: For Inception Resnet V2
input_shape: optional shape tuple, only to be specified if include_top
is False (otherwise the input shape has to be (299, 299, 3) (with
'channels_last' data format) or (3, 299, 299) (with 'channels_first'
data format). It should have exactly 3 inputs channels, and width and
height should be no smaller than 139. E.g. (150, 150, 3) would be one
valid value.
include_top: whether to include the fully-connected layer at the top
of the network.
https://keras.io/applications/#inceptionresnetv2
This is simply because the fully connected layers at the end can only take fixed size inputs, which has been previously defined by the input shape and all processing in the convolutional layers. Any change to the input shape will change the shape of the input to the fully connected layers, making the weights incompatible (matrix sizes don't match and cannot be applied).
This is a specific problem to fully connected layers. If you use another layer for classification, such as global average pooling, then one would not have this problem.

How to get keras layer's output in a multiple-input model?

When a Keras model accept multiple inputs, its layers behave like there is just one input. It might be a bug.
model = vgg19.VGG19(weights='imagenet', include_top=False, pooling='avg')
model(image1)
model(image2)
model.get_output_at(0)
model.get_output_at(1)
#no error here
outputs_0 = [layer.get_output_at(0) for layer in model.layers]
#no error here
outputs_1 = [layer.get_output_at(1) for layer in model.layers]
#error "Asked to get output at node 1, but the layer has only 1 inbound nodes."
I'm really not sure about what is outputs_0, since model have two inputs, image1 and image2, and when a layer return its output, what is its corresponding input?
In keras, If you have a model: .
print your model, you can know layer name;
wrap a new model;
get output;
from keras.models import Model
print(<your_model>.summary())
<new_model> = Model(inputs=<your_model>.input, outputs=<your_model>.get_layer('your layer_name').get_output_at(<index_number>))
<your_output> = <new_model>.predict(<your_input>)
Regardless of the model's inputs and outputs, there is no rule about how the layers behave inside a model. A model may have many internal branches and reuse (or not) the same layer with different inputs, yielding thus different outputs. A layer will only have "output at 1 (or more)" if that layer was used more than once.
The only certain things are:
the input layers will match the model's input (see 1),
and the output layers will match the model's output (see 1).
But anything is possible in between (see 2).
(1) - But, a model that has many inputs/outputs actually has many "input/output layers". Each output layer has a single output. If you check the "model" outputs, you have many, but if you check the "layers" outputs, then there are several output layers, each yealding a single output (output at 0 only). The same is valid for model's inputs vs input layers.
(2) - Even though, the most common option is to have layers being used only once, and thus having only "output at 0", without additional outputs.

Resources