I am trying to use Conv1d as a first layer. I read the link https://keras.io/api/layers/convolution_layers/convolution1d/ but failed to understand.
If I use the first layer as dense like
model.add(keras.layers.Dense(12, input_dim=232, activation='relu'))
it works fine but if try to use the first layer as Conv1d like
model.add(keras.layers.Conv1D(32, 5, activation = 'relu'))
I get an error as:
Input 0 of layer "conv1d" is incompatible with the layer: expected min_ndim=3, found ndim=2. Full shape received: (1, 232)
My input size is 3000X232 and I am trying to learn whether a particular vector is present or not. So my output is either 0 or 1. If the vector is present in the input, the output is 1 and if the vector is absent in the input the output is 0. So I am learning a simple two-class classifier.
Can anyone help? Is it that there must be at least 2 spatial dimensions to use Conv1D?
Related
I am following the self attention in Keras in the following link: How to add attention layer to a Bi-LSTM
I am new to python , what does the shape=(input_shape[-1],1) in self.add_weight and shape=(input_shape[1],1) in bias means?
The shape argument sets the expected input dimensions which the model will be fed. In your case, it is just going to be whatever the last dimension of the input shape is for the weight layer and the second dimension of the input shape for the bias layer.
Neural networks take in inputs of fixed size so while building a model, it is important that you hard code the input dimensions for each layer.
I'm trying to build an autoencoder like network but I don't know how to specify the output shape of the network. I have an input of size mxn and a paired expected output of size pxq. I've seen
Calculate the Output size in Convolution layer
Getting the output shape of deconvolution layer using tf.nn.conv2d_transpose in tensorflow
but is there a way force an output shape without having to work out a bunch of math for every input shape?
I really don't think there is a way to do this (tough I will be happy to learn otherwise), since this output shape of a conv layer of any kind is the result of a mathematical operation (convolution) with several parameters. because of that, the resulting shape has to be one of the possible shapes, based on the input tensor and the parameters (stride, kernel size and so on).
this is in contrast to a dense (fully-connected layer), where you can get any shape you want as long as its a single number (4, 60 or 5000 - but not (60,60)).
one small trick that can help you in this kind of situations sometimes is to get the shape of the previous layer print it so you know what parameters you need for the next layer and make sure your calculations are correct:
import keras.backend as K
x = Conv2D()(x) # or any other layer
shape = K.int_shape(x)
print(shape)
x = Conv2D()(x)
I want to use 3CNN with 3GRU layers. Here is the architecture:
layer_a = Input(shape=(120,), dtype='float32',name='main')
layer_b = Input(shape=(9,), dtype='float32', name='site')
layer_c = Input(shape=(4,), dtype='float32', name='access')
model = Model(inputs=[layer_a, layer_b,layer_c], outputs=[layer_f])
model.compile(optimizer='adam',loss=smape_error)
But when I tried to fit into my data, it produces an error:
Input 0 is incompatible with layer gru_14: expected ndim=3, found ndim=2.
Not sure what went wrong?
GRU layers need the following dimension (batch_size, seq_len, dim_per_seq)
also it returns (batch_siz, number_of_neurons) so in order to put 2 GRU after each other, the first GRU layer need to set the parameter return_sequences=True.
Also, when building keras models it's always a good idea to use model.summary()(just build part of the model before the erroe appears) to debug. Often the problem lies in an unexpected shape.
Your architecture is not suited for using GRU layers at all. First you can't flatten the tensors because this will destroy your sequence like structure. This will make concatenating the layers impossible. You could conv and pool your tree layers layer_t, layer_tt and layer_ttt to the same second dimension (should be bigger then 1). This way you could concatenate the last dimension and get a tensor with a sequence like shape to put into a gru layer.
I have created a CNN in Keras with 12 Convolutional layers each followed by BatchNormalization, Activation and MaxPooling. A sample of the layer is:
model.add(Conv2D(256, (3, 3), padding='same'))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=2))
I start with 32 feature maps and end with 512. If I add MaxPooling after every Conv Layer like in the code above, I get an error in the final layer:
ValueError: Negative dimension size caused by subtracting 2 from 1 for 'max_pooling2d_11/MaxPool' (op: 'MaxPool') with input shapes: [?,1,1,512].
If I omit one MaxPooling in any layer the model compiles and starts training. I am using Tensorflow as backend and I have the right input shape of the image in the first layer.
Are there any suggestions why this may happening?
If your spatial dimensions are 256x256, then you cannot have more than 8 Max-Pooling layers in your network. As 2 ** 8 == 256, after downsampling by a factor of two, eight times, your feature maps will be 1x1 in the spatial dimensions, meaning you cannot perform max pooling as you would get a 0x0 or negative dimensions.
Its just an obvious limitation of Max Pooling but not always discussed in papers.
This can also be caused by having your input image in the wrong format
if you're using (3,X,Y) and it expects (X,Y,3) then the down sampling occurs on the colour channels and causes issues.
My problem is to take all hidden outputs from an LSTM and use them as training examples for a single dense layer. Flattening the output of the hidden layers and feeding them to a dense layer is not what I am looking to do. I have tried the following things:
I have considered Timedistributed wrapper for the dense layer (https://keras.io/layers/wrappers/). But, this seems to apply the same layer to every time slice, which is not what I want. In other words, the Timedistributed wrapper has input_shape of a 3D tensor (number of samples, number of timesteps, number of features) and produces another 3D tensor of the same type: (number of samples, number of timesteps, number of features). Instead what I want is a 2D tensor as output, which looks like (number of samples*number of timesteps, number of features)
There was a pull request for an AdvancedReshapeLayer: https://github.com/fchollet/keras/pull/36 on GitHub. This seems to be exactly what I am looking for. Unfortunately, it appears like that pull request was closed with no conclusive outcome.
I tried to build my own lambda layer to accomplish what I want as follows:
A). model.add(LSTM(NUM_LSTM_UNITS, return_sequences=True, activation='tanh')) #
B). model.add(Lambda(lambda x: x, output_shape=lambda x: (x[0]*x[1], x[2])))
C). model.add(Dense(NUM_CLASSES, input_dim=NUM_LSTM_UNITS))
mode.output_shape after (A) prints: (BATCH_SIZE, NUM_TIME_STEPS, NUM_LSTM_UNITS) and model.output_shape after (B) prints: (BATCH_SIZE*NUM_OF_TIMESTEPS, NUM_LSTM_UNITS)
Which is exactly what I am trying to achieve.
Unfortunately, when I try to run step (C). I get the following error:
Input 0 is incompatible with layer dense_1: expected ndim=2, found
ndim=3
This is baffling since when I print model.output_shape after (B), I do indeed see (BATCH_SIZE*NUM_OF_TIMESTEPS, NUM_LSTM_UNITS), which is of ndim=2.
Really appreciate any help with this.
EDIT: When I try to use the functional API instead of a sequential model, I still get the same error on step (C)
You can use backend reshape which includes batch_size dimension.
def backend_reshape(x):
return backend.reshape(x, (-1, NUM_LSTM_UNITS))
model.add(Lambda(backend_reshape, output_shape=(NUM_LSTM_UNITS,)))