PyTorch model not converging - pytorch

Question

I'm training a binary classification model on a series of images.
The model was derived from resnet18 in torchvision and I made the last FC as nn.Linear(512, 1)
The loss function is BCELoss
However, the model doesn't show any sign of converging even after 5000 iterations.
I'm suspecting I might do something wrong in the training stage? But I can't find where's the bug.
Here's my code:
Model:
## Model
import torch.nn as nn
import torch.nn.functional as F
import torchvision.models as models
resnet18 = models.resnet18(pretrained= True)
resnet18.fc = nn.Linear(512, 1)
Parameters, loss, optimizers:
## parameter
epochs = 200
learning_rate = 0.1
momen = 0.9
batch = 8
criterion = nn.BCELoss()
resnet18.to(device)
opt = optim.SGD(resnet18.parameters(), lr = learning_rate, momentum = momen)
Dataloaders:
# Generators
training_set = Dataset(X_train)
training_generator = torch.utils.data.DataLoader(training_set, batch_size= batch, shuffle=True)
validation_set = Dataset(X_test)
validation_generator = torch.utils.data.DataLoader(validation_set, batch_size=1, shuffle=False)
Training:
# training
history = []
for t in range(epochs):
for i, data in enumerate(training_generator, 0):
inputs, labels = data
# check if input size == batch size #
if inputs.shape[0] < batch:
break
# print("labels", labels, labels.dtype)
# move data to GPU #
inputs, labels = inputs.to(device), labels.to(device)
opt.zero_grad()
# Prediction #
y_pred = resnet18(inputs).view(batch,)
y_pred = (y_pred > 0).float().requires_grad_()
# print("y_pred", y_pred, y_pred.dtype)
# Calculating loss #
loss = criterion(y_pred, labels.view(batch,))
loss.backward()
opt.step()
if i % 10 == 0:
history.append(loss.item())
print("Epoch: {}, iter: {}, loss: {}".format(t, i, loss.item())
torch.save(resnet18, 'trained_resnet18.pt')
Edit:
The loss values are like this:
Epoch: 3, iter: 310, loss: 0.0
Epoch: 3, iter: 320, loss: 37.5
Epoch: 3, iter: 330, loss: 37.5
Epoch: 3, iter: 340, loss: 0.0
Epoch: 3, iter: 350, loss: 37.5
Epoch: 3, iter: 360, loss: 50.0
Epoch: 3, iter: 370, loss: 37.5
Epoch: 3, iter: 380, loss: 25.0
Epoch: 3, iter: 390, loss: 12.5

Answer 1

I belive the error lies in the following line:
y_pred = (y_pred > 0).float().requires_grad_()
You try to binarize the model prediction in a weird way, I suggest do the following instead:
y_pred = torch.sigmoid(y_pred)
And pass this to the loss function.
Explanation
The output of the model can be any value, but we want to normalize that values to reside in the [0,1] range. This is exactly what the sigmoid function does. Once we have the values in the range of [0,1] the comparison with the binary labels will make sense, closer to 1 will be "1" and the opposite.
You can refer to the following link: https://www.youtube.com/watch?v=WsFasV46KgQ