Given a below code in PyTorch what would be the Keras equivalent?
class Network(nn.Module):
def __init__(self, state_size, action_size):
super(Network, self).__init__()
# Inputs = 5, Outputs = 3, Hidden = 30
self.fc1 = nn.Linear(5, 30)
self.fc2 = nn.Linear(30, 3)
def forward(self, state):
x = F.relu(self.fc1(state))
outputs = self.fc2(x)
return outputs
Is it this?
model = Sequential()
model.add(Dense(units=30, input_dim=5, activation='relu'))
model.add(Dense(units=30, activation='relu'))
model.add(Dense(units=3, activation='linear'))
or is it this?
model = Sequential()
model.add(Dense(units=30, input_dim=5, activation='linear'))
model.add(Dense(units=30, activation='relu'))
model.add(Dense(units=3, activation='linear'))
or is it?
model = Sequential()
model.add(Dense(units=30, input_dim=5, activation='relu'))
model.add(Dense(units=30, activation='linear'))
model.add(Dense(units=3, activation='linear'))
Thanks
None of them looks correct according to my knowledge. A correct Keras equivalent code would be:
model = Sequential()
model.add(Dense(30, input_shape=(5,), activation='relu'))
model.add(Dense(3))
model.add(Dense(30, input_shape=(5,), activation='relu'))
Model will take as input arrays of shape (*, 5) and output arrays of shape (*, 30). Instead of input_shape, you can use input_dim also. input_dim=5 is equivalent to input_shape=(5,).
model.add(Dense(3))
After the first layer, you don't need to specify the size of the input anymore. Moreover, if you don't specify anything for activation, no activation will be applied (equivalent to linear activation).
Another alternative would be:
model = Sequential()
model.add(Dense(30, input_dim=5))
model.add(Activation('relu'))
model.add(Dense(3))
Hopefully this makes sense!
Looks like a
model = Sequential()
model.add(InputLayer(input_shape=input_shape(5,))
model.add(Dense(30, activation='relu')
model.add(Dense(3))
If you are trying to convert Pytorch model to Keras model, you can also try a Pytorch2Keras converter.
It supports base layers like Conv2d, Linear, Activations, Element-wise operations. So, I've converted ResNet50 with error 1e-6.
model = Sequential()
model.add(Dense(30, input_dim=5, activation='relu'))
model.add(Dense(3, activation=None))
Related
I'm trying to construct an encoder to get the latent space in order to plot it. I don't really know if I can get it from the RepeatVector or if I have to add a Dense layer.
Here is my code:
model = Sequential()
model.add(LSTM(16, activation='relu', return_sequences=True, input_shape= (x_train.shape[1], 1)))
model.add(LSTM(4, activation='relu', return_sequences=False)) #Encoder
model.add(RepeatVector(X_train.shape[1])) #Latent
model.add(LSTM(4, activation='relu', return_sequences=True)) #Decoder
model.add(LSTM(16, activation='relu', return_sequences=False)) #Decoder
model.add(TimeDistributed(Dense(X_train.shape[2]))) #Decoder
You need to separate the model into two parts (encoder and decoder).
Then, build the encoder using the input and output of the encoder part.
btw the output would be the last layer before using RepeatVector.
encoder = Model(inputs, output_from_encoder)
i am using fashion_mnist images database (60,000 small square 28×28 pixel grayscale images) and i am trying to apply CNN-LSTM in cascading, this is the code i am using:
from tensorflow.keras.datasets import fashion_mnist
(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()
print("Shape of x_train: {}".format(x_train.shape))
print("Shape of y_train: {}".format(y_train.shape))
print()
print("Shape of x_test: {}".format(x_test.shape))
print("Shape of y_test: {}".format(y_test.shape))
# define CNN model
model = Sequential()
model.add(TimeDistributed(Conv2D(32, kernel_size=(3, 3),
activation='relu',
input_shape=(60000,28,28))))
model.add(TimeDistributed(Conv2D(64, (3, 3), activation='relu')))
model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))
model.add(TimeDistributed((Dropout(0.25))))
model.add(TimeDistributed(Flatten()))
## LSTM
model.add(LSTM(200, activation='relu', return_sequences=True))
model.add(Dense(128, activation='relu'))
model.compile(loss='categorical_crossentropy', optimizer='adam',
metrics=['accuracy'])
##fitting model
model.fit(x_train,y_train,epochs=5)
test_loss, test_acc=model.evaluate(x_test,y_test)
print('Loss: {0}-Acc:{1}')
print(test_acc)
i get the error after running the fitting line, can any one help me solving the error.
Define an Input layer with a four-channel output instead of defining the input with Conv2D layer.
I have a code below which implements an architecture (in grid search), to yield appropriate parameters for input, nodes, epochs, batch size and differenced time series input.
The challenge I have is to convert the neural network from just having one LSTM hidden layer, to multiple LSTM hidden layers.
At the moment, I could only run the code with Dense-type hidden layers, without having any errors thrown, otherwise I get dimension errors, tuple errors and so on.
The problem is only persistent in the neural network architecture section.
Original code that works:
def model_fit(train, config):
# unpack config
n_input, n_nodes, n_epochs, n_batch, n_diff = config
# Data
if n_diff > 0:
train = difference(train, n_diff)
# Time series to supervised format
data = series_to_supervised(train, n_in=n_input)
train_x, train_y = data[:, :-1], data[:, -1]
# Reshaping input data into [samples, timesteps, features]
n_features = 1
train_x = train_x.reshape((train_x.shape[0], train_x.shape[1], n_features))
# Define model for (Grid search architecture)
model = Sequential()
model.add(LSTM(n_nodes, activation='relu', input_shape=(n_input, n_features)))
model.add(Dense(n_nodes, activation='relu'))
model.add(Dense(n_nodes, activation='relu'))
model.add(Dense(n_nodes, activation='relu'))
model.add(Dense(1))
# Compile model (Grid search architecture)
model.compile(loss='mse', optimizer='adam')
# fit model
model.fit(train_x, train_y, epochs=n_epochs, batch_size=n_batch, verbose=0)
return model
Modified LSTM-hidden layer code, that fails to run:
# Define model for (Grid search architecture)
model = Sequential()
model.add(LSTM(n_nodes, activation='relu', input_shape=(n_input, n_features), return_sequences=True))
model.add(LSTM(n_nodes, activation='relu', return_sequences=True))
model.add(LSTM(n_nodes, activation='relu', return_sequences=True))
model.add(LSTM(n_nodes, activation='relu', return_sequences=True))
model.add(TimeDistributed(Dense(1)))
Another variant that also threw an error - ValueError: Error when checking target: expected time_distributed_4 to have 3 dimensions, but got array with shape (34844, 1)
model = Sequential()
model.add(LSTM(n_nodes, activation='relu', input_shape=(n_input, n_features), return_sequences=True))
model.add(LSTM(n_nodes, activation='relu', return_sequences=False))
model.add(RepeatVector(n_input))
model.add(LSTM(n_nodes, activation='relu', return_sequences=True))
model.add(LSTM(n_nodes, activation='relu', return_sequences=True))
model.add(TimeDistributed(Dense(n_features)))
Could anyone with any suggestion please help me ?
Try to set return_sequences=False at the last layer.
Combine two models DropBlock(dimesnions=4), and CuDNNGRU(dimension=3) for MNIST image Classification. How can I combine different dimensions of Drop Block and GRU models?
GRU mostly used for Document classification but here I'm trying to classify MNIST dataset. Both Models separately score ~98%. I want to combine both DropBlock and GRU Model in a single classification Model.
The codes of combining the DropBlock and GRU model for MNIST classification.
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train/255.0
x_test = x_test/255.0
print(x_train.shape)
print(x_train[0].shape)
#load the model
model = Sequential()
gru = Sequential()
gru.add(CuDNNGRU(128,input_shape=(x_train.shape[1:]),
return_sequences=True))
gru.add(BatchNormalization())
gru.add(CuDNNGRU(128))
gru.add(BatchNormalization())
print(gru.output_shape)
drop = Sequential()
drop.add(DropBlock2D(input_shape=(28, 28, 1), block_size=7,
keep_prob=0.8, name='Input-Dropout'))
drop.add(Conv2D(filters=64, kernel_size=3, activation='relu',
padding='same', name='Conv-1'))
drop.add(MaxPool2D(pool_size=2, name='Pool-1'))
drop.add(DropBlock2D(block_size=5, keep_prob=0.8, name='Dropout-1'))
drop.add(Conv2D(filters=32, kernel_size=3, activation='relu',
padding='same', name='Conv-2'))
drop.add(MaxPool2D(pool_size=2, name='Pool-2'))
print(drop.output_shape)
model.add(merge([drop, gru], mode='concat', concat_axis=2))
print(model.output_shape)
model.add(Flatten(name='Flatten'))
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(10, activation='softmax'))
score = model.evaluate(x_test, y_test, verbose=0)
The expected result ~98% results on this classification model.
I'm trying to use maxpooling as a first layer using keras and I have a problem with the input and output dimensions.
print(x_train.shape)
print(y_train.shape)
(15662, 6)
(15662,)
x_train = np.reshape(x_train, (-1,15662, 6))
y_train = label_array.reshape(1, -1)
model = Sequential()
model.add(MaxPooling1D(pool_size = 2 , strides=1, input_shape = (15662,6)))
model.add(Dense(5, activation='relu'))
model.add(Flatten())
model.add(Dense(1, activation='softmax'))
model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=
['accuracy'])
model.fit(x_train, y_train, batch_size= 32, epochs=1)
After running the model, I get the following error:
ValueError: Error when checking target: expected dense_622 (last layer)
to have shape (1,) but got array with shape (15662,)
I'm doing classification and my target is binary (0,1)
Thank you
Your target should have shape (batch_size, 1) but you are passing an array of shape (1, 15662). It seems like 15662 should be the batch size, in which case x_train should have shape (15662, 6) and y_train should have shape (15662, 1). In this case however, it doesn't make any sense to have a MaxPooling1D layer as the first layer of your model since max pooling requires a 3D input (i.e. shape (batch_size, time_steps, features)). You probably want to leave out the max pooling layer (and the Flatten layer). The following code should work:
# x_train: (15662, 6)
# y_train: (15662,)
model = Sequential()
model.add(Dense(5, activation='relu', input_shape=(6,))) # Note: don't specify the batch size in input_shape
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=
['accuracy'])
model.fit(x_train, y_train, batch_size= 32, epochs=1)
But it of course depends on what kind of data you have.