I'm trying to add a RNN layer within a Convolutional layer. Unfortunately due to difference of ndim it's failing to create a model.
Model:
model = keras.Sequential(
[
# layers.Rescaling(1.0/255),
keras.Input(shape=(256, 256, 3)),
layers.Conv2D(32, (3,3), padding="valid", activation='swish'),
layers.MaxPooling2D(pool_size=(2,2)),
layers.Conv2D(64, 3, activation="swish"),
layers.MaxPooling2D(),
layers.Conv2D(32, 3, activation="swish"),
layers.Flatten(),
layers.Dense(64, activation='sigmoid'),
layers.Dense(10),
layers.Dense(2),
]
)
It'll be really helpful, if someone can help to figure this out :)
EDIT
Code that gave the error
model = keras.Sequential(
[
# layers.Rescaling(1.0/255),
keras.Input(shape=(256, 256, 3)),
layers.Conv2D(32, (3,3), padding="valid", activation='swish'),
layers.SimpleRNN(512, activation='relu')
layers.MaxPooling2D(pool_size=(2,2)),
layers.Conv2D(64, 3, activation="swish"),
layers.MaxPooling2D(),
layers.Conv2D(32, 3, activation="swish"),
layers.Flatten(),
layers.Dense(64, activation='sigmoid'),
layers.Dense(10),
layers.Dense(2),
]
)
Error Message
Input 0 of layer "simple_rnn" is incompatible with the layer: expected ndim=3, found ndim=4. Full shape received: (None, 254, 254, 32)
If you add a layers.Reshape layer before and after the RNN layer, the shape issue is resolved.
model = keras.Sequential(
[
keras.Input(shape=(256, 256, 3)),
layers.Conv2D(32, (3,3), padding="valid", activation='swish'),
layers.Reshape((-1, 32)), # flatten only the two spatial dimensions
layers.SimpleRNN(512),
layers.Reshape((16, 16, 2)), # whatever shape you like
layers.MaxPooling2D(pool_size=(2,2)),
layers.Conv2D(64, 3, activation="swish"),
layers.MaxPooling2D(),
layers.Conv2D(32, 3, activation="swish"),
layers.Flatten(),
layers.Dense(64, activation='sigmoid'),
layers.Dense(10),
layers.Dense(2),
]
)
I don't know if it makes sense to use 2d convolution after the RNN layer. I think it might destroy the spatial semantics of the input. Also the RNN layer will have a huge number of weights. That may speak for adding the RNN layer at the end. Reshaping will work there as well, but you will have to add a dimension, instead of flattening one.
Related
I wanted to create a visual diagram for my neural network. I started with a LeNet style using NN-SVG to diagram my convolutional neural network, but I wasn't sure which numbers to plug in where. The filters for my layers double each layer, whilst filter size stays static at 3*3.
Below is the code for the CNN, it was made for the CIFAR-10 dataset.
model = Sequential([
layers.Conv2D(32, (3, 3), padding = 'same', activation = 'relu', input_shape = (32, 32, 3)),
layers.MaxPooling2D(),
layers.Conv2D(64, (3, 3), padding = 'same', activation = 'relu'),
layers.MaxPooling2D(),
layers.Conv2D(128, (3, 3), padding = 'same', activation = 'relu'),
layers.MaxPooling2D(),
layers.Flatten(),
layers.Dense(128, activation = 'relu'),
layers.Dense(10)
])
Attempted Diagram doesn't look quite right, I've put the depth of each layer as the amount of nodes in that layer, but is this correct or should it be 3 for the amount of channels?
I am trying to build a Q-learning model using Reinforcement Learning to play a simple "Chrome Dino Game". But I got stuck when trying to use an OpenCV function to filter the images effortlessly. This here is my CNN code:
class ConvolutionalNeuralNetwork:
def __init__(input_shape, action_space): # Removed Self
model = models.Sequential()
model.add(Conv2D(32,
8,
strides=(4, 4),
padding="valid",
activation="relu",
input_shape=input_shape,
data_format="channels_first"))
model.add(Conv2D(64,
4,
strides=(2, 2),
padding="valid",
activation="relu",
input_shape=input_shape,
data_format="channels_first"))
model.add(Conv2D(64,
3,
strides=(1, 1),
padding="valid",
activation="relu",
input_shape=input_shape,
data_format="channels_first",))
model.add(Flatten())
model.add(Dense(512, activation="relu"))
model.add(Dense(action_space))
model.compile(loss="mean_squared_error",
optimizer=RMSprop(lr=0.00025,
rho=0.95,
epsilon=0.01),
metrics=["accuracy"])
return cur_input
model.build(input_shape=(80,80))
model.summary()
I had spied this simple function to efficiently preprocess my output to the desired level however am not able to figure out how and where to connect them both in the code
def process_img(image):
image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
image = image[:300, :500]
return image
For a clear demonstration, I am trying to do something like this (ignore the last set):
Note: I will prefer to use OpenCV if possible but if there is some other alternative which is FAST and efficient and can be easily used, I shall accept it
I tried to implement 1d convolution with dilation
#keras.layers.Conv1D(filters, kernel_size, strides=1, padding='valid', data_format='channels_last', dilation_rate=1, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)
# valid , causal , same
conv = layers.Conv1D(1, 3, padding='same',
dilation_rate=1,
bias_initializer=tf.keras.initializers.zeros)
I want to understand how this dilation 1d convolution actually thraw the output
let say we have a input with
np.squeeze(sequence.numpy())
array([0. , 0.32380696, 0.61272254, 0.83561502, 0.96846692])
and we have convolution filter with
np.squeeze(conv.trainable_variables[0].numpy())
array([-0.56509803, 0.89481053, 0.6975754 ])
and when we pass the convolution the output will be
output = conv(sequence)
np.squeeze(output.numpy())
array([0. , 0.22587977, 0.71716606, 0.94819239, 1.07704752])
try to implement the wavenet 1dconvolution with dilation
I want to know how this output value calculated
and what if filter size and kernel_size change to different number?
conv = layers.Conv1D(2, 3, padding='causal',
dilation_rate=1,
bias_initializer=tf.keras.initializers.zeros)
conv = layers.Conv1D(3, 3, padding='causal',
dilation_rate=1,
bias_initializer=tf.keras.initializers.zeros)
conv = layers.Conv1D(1, 3, padding='causal',
dilation_rate=2,
bias_initializer=tf.keras.initializers.zeros)
conv = layers.Conv1D(2, 3, padding='same',
dilation_rate=1,
bias_initializer=tf.keras.initializers.zeros)
conv = layers.Conv1D(3, 3, padding='same',
dilation_rate=1,
bias_initializer=tf.keras.initializers.zeros)
conv = layers.Conv1D(1, 3, padding='same',
dilation_rate=2,
bias_initializer=tf.keras.initializers.zeros)
I am playing with a model which should take a 8x8 chess board as input, encoded as a 224x224 grey image, and then output a 64x13 one-hot-encoded logistic regression = probabilities of pieces on the squares.
Now, after the Convolutional layers I don't quite know, how to proceed to get a 2D-Dense layer as a result/target.
I tried adding a Dense(64,13) as a layer to my Sequential model, but I get the error "Dense` can accept only 1 positional arguments ('units',)"
Is it even possible to train for 2D-targets?
EDIT1:
Here is the relevant part of my code, simplified:
# X.shape = (10000, 224, 224, 1)
# Y.shape = (10000, 64, 13)
model = Sequential([
Conv2D(8, (3,3), activation='relu', input_shape=(224, 224, 1)),
Conv2D(8, (3,3), activation='relu'),
# some more repetitive Conv + Pooling Layers here
Flatten(),
Dense(64,13)
])
TypeError: Dense can accept only 1 positional arguments ('units',), but you passed the following positional arguments: [64, 13]
EDIT2: As Anand V. Singh suggested, I changed Dense(64, 13) to Dense(832), which works fine. Loss = mse.
Wouldn't it be better to use "sparse_categorical_crossentropy" as loss and 64x1 encoding (instead of 64x13) ?
In Dense you only pass the number of layers you expect as output, if you want (64x13) as output, put the layer dimension as Dense(832) (64x13 = 832) and then reshape later. You will also need to reshape Y so as to accurately calculate loss, which will be used for back propagation.
# X.shape = (10000, 224, 224, 1)
# Y.shape = (10000, 64, 13)
Y = Y.reshape(10000, 64*13)
model = Sequential([
Conv2D(8, (3,3), activation='relu', input_shape=(224, 224, 1)),
Conv2D(8, (3,3), activation='relu'),
# some more repetitive Conv + Pooling Layers here
Flatten(),
Dense(64*13)
])
That should get the job done, if it doesn't post where it fails and we can proceed further.
A Reshape layer allows you to control the output shape.
Flatten(),
Dense(64*13),
Reshape((64,13))#2D
I am pretty new to deep learning; I want to train a network on image patches of size (256, 256, 3) to predict three labels of pixel-wise segmentation. As a start I want to provide one convolutional layer:
model = Sequential()
model.add(Convolution2d(32, 16, 16, input_shape=(3, 256, 256))
The model output so far is an image with 32 channels. Now, I want to add a dense layer which merges all of these 32 channels into three channels, each one predicting the probability of a class for one pixel.
How can I do that?
The simplest method to merge your 32 channels back to 3 would be to add another convolution, this time with three filters (I arbitrarily set the filter sizes to be 1x1):
model = Sequential()
model.add(Convolution2d(32, 16, 16, input_shape=(3, 256, 256))
model.add(Convolution2d(3, 1, 1))
And then finally add an activation function for segmentation
model.add(Activation("tanh"))
Or you could add it all at once if you want to with activation parameter (arbitrarily chosen to be tanh)
model = Sequential()
model.add(Convolution2d(32, 16, 16, input_shape=(3, 256, 256))
model.add(Convolution2d(3, 1, 1,activation="tanh"))
https://keras.io/layers/convolutional/
You have to use flatten between the convolution layers and the dense layer:
model = Sequential()
model.add(Convolution2d(32, 16, 16, input_shape=(3, 256, 256))
# Do not forget to add an activation layer after your convolution layer, so here.
model.add(Flatten())
model.add(Dense(3))
model.add(Activation("sigmoid")) # whatever activation you want.