I am trying to overfit my model on a single batch to check model integrity. I am using Keras and TensorFlow for the implementation of my model and coding style for this project.
I know how to get the single batch and overfit the model in PyTorch but don't have an idea in Keras.
to get a single batch in PyTorch I used:
images, labels = next(iter(train_dataset))
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr = 0.0001)
for epoch in range(epochs):
print(f"Epoch [{epoch}/{epochs}]")
# for batch_idx, (data, target) in enumerate(train_loader):
data, target = data.to(device), target.to(device)
data = data.reshape(data.shape[0], -1)
# forward
score = model(data)
loss = criterion(score, target)
print(f"Loss: {loss.item()}")
# backward
optimizer.zero_grad()
loss.backward()
optimizer.step()
How to do it in keras any helping matrial?
Thank you everyone for coming here. I found a solution and here it is:
datagen = ImageDataGenerator(rescale=1 / 255.0,
rotation_range=20,
zoom_range=0.2,
width_shift_range=0.05,
height_shift_range=0.05,
shear_range=0.2,
horizontal_flip=True,
fill_mode="nearest"
)
# preprocessing_function=preprocess_input,
# Declare an image generator for validation & testing without generation
test_datagen = ImageDataGenerator(rescale = 1./255,)#preprocessing_function=preprocess_input
# Declare generators for training, validation, and testing from DataFrames
train_gen = datagen.flow_from_directory(directory_train,
target_size=(512, 512),
color_mode='rgb',
batch_size=BATCH_SIZE,
class_mode='binary',
shuffle=True)
val_gen = test_datagen.flow_from_directory(directory_val,
target_size=(512, 512),
color_mode='rgb',
batch_size=BATCH_SIZE,
class_mode='binary',
shuffle=False)
test_gen = test_datagen.flow_from_directory(directory_test,
target_size=(512, 512),
color_mode='rgb',
batch_size=BATCH_SIZE,
class_mode='binary',
shuffle=False)
train_images, train_labels = next(iter(train_gen))
val_images, val_labels = next(iter(val_gen))
test_images, test_labels = next(iter(val_gen))
#check shape for selected Batch
print("Length of Train images : {}".format(len(train_images)))
print("shape of Train images : {}".format(train_images.shape))
print("shape of Train labels : {}".format(train_labels.shape))
Length of Train images : 32
shape of Train images : (32, 512, 512, 3)
shape of Train labels : (32,)
history = model.fit(train_images, train_labels,
use_multiprocessing=True,
workers=16,
epochs=100,
class_weight=class_weights,
validation_data=(val_images, val_labels),
shuffle=True,
callbacks=call_backs)
Related
I am used to work with tenserflow - keras but now I am forced to start working with Pytorch for flexibility issues. However, I don't seem to find a pytorch code that is focused on training only the classifciation layer of a model. Is that not a common practice ? Now I have to wait out the calculation of the feature extraction of the same data for every epoch. Is there a way to avoid that ?
# in tensorflow - keras :
from tensorflow.keras.applications import vgg16, MobileNetV2, mobilenet_v2
# Load a pre-trained
pretrained_nn = MobileNetV2(weights='imagenet', include_top=False, input_shape=(Image_size, Image_size, 3))
# Extract features of the training data only once
X = mobilenet_v2.preprocess_input(X)
features_x = pretrained_nn.predict(X)
# Save features for later use
joblib.dump(features_x, "features_x.dat")
# Create a model and add layers
model = Sequential()
model.add(Flatten(input_shape=features_x.shape[1:]))
model.add(Dense(100, activation='relu', use_bias=True))
model.add(Dense(Y.shape[1], activation='softmax', use_bias=False))
# Compile & train only the fully connected model
model.compile( loss="categorical_crossentropy", optimizer=keras.optimizers.Adam(learning_rate=0.001))
history = model.fit( features_x, Y_train, batch_size=16, epochs=Epochs)
Assuming you already have the features ìn features_x, you can do something like this to create and train the model:
# create a loader for the data
dataset = torch.utils.data.TensorDataset(features_x, Y_train)
loader = torch.utils.data.DataLoader(dataset, batch_size=16, shuffle=True)
# define the classification model
in_features = features_x.flatten(1).size(1)
model = torch.nn.Sequential(
torch.nn.Flatten(),
torch.nn.Linear(in_features=in_features, out_features=100, bias=True),
torch.nn.ReLU(),
torch.nn.Linear(in_features=100, out_features=Y.shape[1], bias=False) # Softmax is handled by CrossEntropyLoss below
)
model.train()
# define the optimizer and loss function
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
loss_function = torch.nn.CrossEntropyLoss()
# training loop
for e in range(Epochs):
for batch_x, batch_y in enumerate(loader):
optimizer.zero_grad() # clear gradients from previous batch
out = model(batch_x) # forward pass
loss = loss_function(out, batch_y) # compute loss
loss.backward() # backpropagate, get gradients
optimizer.step() # update model weights
I have a Multi-Layer Perceptron network in Keras with two hidden Layers.
While trying to train the network I get the Error in the fit_generator :
Error:
ValueError: logits and labels must have the same shape ((None, 2) vs (None, 1))
My Code is:
import numpy as np
import keras
from keras import layers
from keras import Sequential
# Define Window size (color images)
img_window = (32,32,3)
# Flatten the Window shape
input_shape = np.prod(img_window)
print(input_shape)
# Define MLP with two hidden layers(neurons)
simpleMLP = Sequential(
[layers.Input(shape=img_window),
layers.Flatten(), # Flattens the input, conv2D to 1 vector , which does not affect the batch size.
layers.Dense(input_shape//2 ,activation="relu"),
layers.Dense(input_shape//2 ,activation="relu"),
layers.Dense(2,activation="sigmoid"),
]
)
# After model is "built" call its summary() menthod to display its contents
simpleMLP.summary()
# Initialization
# Size of the batches of data, adjust it depends on RAM
batch_size = 128
epochs = 5
# Compile MLP model with 3 arguments: loss function, optimizer, and metrics function to judge model performance
simpleMLP.compile(loss="binary_crossentropy",optimizer="adam",metrics=["binary_accuracy"]) #BCE
# Create ImagedataGenerator to splite training, validation dataset
from keras.preprocessing.image import ImageDataGenerator
train_dir = '/content/train'
train_datagen = ImageDataGenerator(
rescale=1./255, # rescaling factor
shear_range=0.1,
zoom_range=0.1,
horizontal_flip=True,
fill_mode='nearest')
valid_dir = '/content/valid'
valid_datagen =ImageDataGenerator(
rescale=1./255,
shear_range=0.1,
zoom_range=0.1,
horizontal_flip=True,
fill_mode='nearest')
train_generator = train_datagen.flow_from_directory(
train_dir,
target_size=img_window[:2],
batch_size=batch_size,
class_mode='binary',
color_mode='rgb'
)
validation_generator = valid_datagen.flow_from_directory(
valid_dir,
target_size=img_window[:2],
batch_size=batch_size,
class_mode='binary',
color_mode='rgb')
# Train the MLP model
simpleMLP.fit_generator((
train_generator,
steps_per_epoch= 8271 // batch_size,
epochs=5,
validation_data=validation_generator,
validation_steps= 2072 // batch_size)
Can you please advise me how to resolve this problem? thanks in advance.
You problem simply is that, you have got labels of shape (N, 1) and loss defined as binary_crossentropy. This means you should have a single output node in the last layer. But you have a model that outputs two classes.
simpleMLP = Sequential(
[...
layers.Dense(2,activation="sigmoid"),
]
)
Simply change this to,
simpleMLP = Sequential(
[...
layers.Dense(1,activation="sigmoid"),
]
)
I have train_x and valid_x splited from trainX ,train_y and valid_y splited from trainY and they are having shapes as per below. i want to classify images of labels LABELS = set(["Faces", "Leopards", "Motorbikes", "airplanes"]).
print(train_x.shape, len(train_y))
torch.Size([1339, 96, 96, 3]) 1339
print(valid_x.shape, len(valid_y))
torch.Size([335, 96, 96, 3]) 335
print(testX.shape, len(testY))
torch.Size([559, 96, 96, 3]) 559
so i want to use regular train/valid on data batch-wise code as per below :
#train the network
n_epochs = 20
valid_loss = []
train_loss = []
for epoch in range(1,n_epochs+1):
cur_train_loss = 0.0
cur_valid_loss = 0.0
#####################
#### Train model ####
#####################
cnn_model.train()
for data, target in trainLoader:
if train_on_gpu:
data, target = data.cuda(), target.cuda()
optimizer.zero_grad()
output = cnn_model(data)
loss = criterion(output, target)
loss.backward()
optimizer.step()
cur_train_loss += loss.item() * data.size(0)
########################
#### Validate model ####
########################
cnn_model.eval()
for data, target in validLoader:
if train_on_gpu:
data, target = data.cuda(), target.cuda()
output = cnn_model(data)
loss = criterion(output, target)
cur_valid_loss += loss.item() * data.size(0)
# calculate avg loss
avg_train_loss = cur_train_loss / len(trainLoader.sampler)
avg_valid_loss = cur_valid_loss / len(validLoader.sampler)
train_loss.append(avg_train_loss)
valid_loss.append(avg_valid_loss)
print('Epoch: {} \t train_loss: {:.6f} \t valid_loss: {:.6f}'.format(epoch, avg_train_loss, avg_valid_loss))
so what i have to do for that ?
i have search for that but nothing specific i found out. i want to use pytorch for this. i have built model for another problem same like this but in that i have used DataLoader for loading one batch of data at a time for training and validation.
You can create a dataset with torch.utils.data.TensorDataset, where each sample of train_x is associated with its corresponding label in train_y, such that the DataLoader can create batches as you are used to.
from torch.utils.data import DataLoader, TensorDataset
train_dataset = TensorDataset(train_x, train_y)
train_dataloader = DataLoader(train_dataset, batch_size=BATCH_SIZE, shuffle=True)
valid_dataset = TensorDataset(valid_x, valid_y)
valid_dataloader = DataLoader(valid_dataset, batch_size=BATCH_SIZE, shuffle=False)
test_dataset = TensorDataset(testX, testY)
test_dataloader = DataLoader(test_dataset, batch_size=BATCH_SIZE, shuffle=False)
I'm using keras to build an image classification model and all of my data is in one directory with subfolders for each class:
data/
----class1/
----class2/
----class3/
----class4/
I'm wondering if there is a way to split my data into a train, test AND validation set in keras using ImageDataGenerator.
Yes, there is a way you can do that.
train_datagen = ImageDataGenerator(rescale=1./255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
validation_split=0.2) # set validation split
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='binary',
subset='training') # set as training data
validation_generator = train_datagen.flow_from_directory(
train_data_dir, # same directory as training data
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='binary',
subset='validation') # set as validation data
model.fit_generator(
train_generator,
steps_per_epoch = train_generator.samples // batch_size,
validation_data = validation_generator,
validation_steps = validation_generator.samples // batch_size,
epochs = nb_epochs)
https://keras.io/preprocessing/image/
I passed two days trying to use Neural Structured language to adapt into me CNN Model I use ImageDataGenerator and flow_from_directory when I use model.fit_generator I got an error message:
ValueError:
When passing input data as arrays, do not specify
steps_per_epoch/steps argument. Please use batch_size instead.
I use Keras 2.3.1 and TensorFlow 2.0 as backend
This is a snipped of my code:
num_classes = 4
img_rows, img_cols = 224, 224
batch_size = 16
train_datagen = ImageDataGenerator(
rescale=1./255,
rotation_range=30,
width_shift_range=0.3,
height_shift_range=0.3,
horizontal_flip=True,
fill_mode='nearest')
validation_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_rows, img_cols),
batch_size=batch_size, shuffle=True,
class_mode='categorical')
validation_generator = validation_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_rows, img_cols),
batch_size=batch_size, shuffle=True,
class_mode='categorical')
def vgg():
model1 = Sequential([ ])
return model1
base_model = vgg()
I adapte Datagenerated from (x,y) format to a dictionary format
def convert_training_data_generator():
for x ,y in train_generator:
return {'feature': x, 'label':y}
def convert_testing_data_generator():
for x ,y in validation_generator:
return {'feature': x, 'label': y}
adv_config = nsl.configs.make_adv_reg_config(multiplier=0.2, adv_step_size=0.05)
model = nsl.keras.AdversarialRegularization(base_model, adv_config=adv_config)
train= convert_training_data_generator()
test= convert_testing_data_generator()
history = model.fit_generator(train,
steps_per_epoch= nb_train_samples // batch_size,
epochs = epochs,
callbacks = callbacks,
validation_data = test,
validation_steps = nb_validation_samples // batch_size)
I think here there is the same error. Maybe you should consider using instead model.fit() function. You should define in that case your train input your train labels and the batch_size. In order to figure out the difference between fit and fit_generator, you can follow that link.