Pytorch - mass classification - pytorch

Question

I trained a model to classify picture into 2 classes.
I wanna use it to order my iPhone images, I have got a lot. So I moved the trained model on my MacBook and started some tests.
Everything works as expected as long a the image folder contains not more than 223 pictures.
It seems to have something to do with the size of the dataset, I can reproduce the issue with the following code.
I tried to split the pictures into batches but I could not figure out how. What is the common fix to that issue? Is it possible to use a Dataloader?
test_dataset = ImageFolderWithPaths('/Users/daniel/Gimp/test-ds/test/', test_trans)
# check images
for img, label, data in test_dataset:
print(img.shape, label, data)
data_path = '/Users/daniel/Gimp/test-ds/test/'
data_path = '/Users/daniel/Pictures/Photos1/'
data_file = data.split('/')[-1]
data_file_new = 'class0_' + data.split('/')[-1]
print(data_path + data_file_new)
Output
torch.Size([3, 64, 64]) 0 /Users/daniel/Gimp/test-ds/test/3/020B1786-E77D-47E8-B0F1-DD72313546B7.jpeg
/Users/daniel/Pictures/Photos1/class0_020B1786-E77D-47E8-B0F1-DD72313546B7.jpeg
torch.Size([3, 64, 64]) 0 /Users/daniel/Gimp/test-ds/test/3/020BC6D0-2AC0-4BF3-A540-6630BA7200ED.jpeg
/Users/daniel/Pictures/Photos1/class0_020BC6D0-2AC0-4BF3-A540-6630BA7200ED.jpeg
torch.Size([3, 64, 64]) 0 /Users/daniel/Gimp/test-ds/test/3/021453CC-CAC1-4546-8188-20739F3C778F.jpeg
/Users/daniel/Pictures/Photos1/class0_021453CC-CAC1-4546-8188-20739F3C778F.jpeg
torch.Size([3, 64, 64]) 0 /Users/daniel/Gimp/test-ds/test/3/0235EEF0-2831-46D3-8889-8E57CDF42205.jpeg
/Users/daniel/Pictures/Photos1/class0_0235EEF0-2831-46D3-8889-8E57CDF42205.jpeg
torch.Size([3, 64, 64]) 0 /Users/daniel/Gimp/test-ds/test/3/02398B72-6F73-49D6-843B-25E2A2593DA9.jpeg
/Users/daniel/Pictures/Photos1/class0_02398B72-6F73-49D6-843B-25E2A2593DA9.jpeg
Prediction Method
class pred01():
def __init__(self, imgpath, model):
cc = 0
cc1 = 0
cc0 = 0
cc2 = 0
"""Predict image class"""
self.ds = ImageFolderWithPaths(imgpath, transform=test_trans)
for img, label, data in self.ds:
self.xb = to_device(img.unsqueeze(0), device)
self.yb = model(self.xb)
_, self.preds = torch.max(self.yb, dim=1)
self.preds1 = torch.softmax(self.yb, dim=1).max()
cc += 1
if dataset.classes[self.preds[0].item()] == '0' and self.preds1.item() > 0.80 or self.preds1.item() == 1.00:
#print('normal pic')
cc0 += 1
# pass
elif dataset.classes[self.preds[0].item()] == '1' and self.preds1.item() > 0.80 or self.preds1.item() == 1.00:
#print('class pic, new name: ' 'class1_' + data.split('/')[-1])
#os.rename(data, '/Users/daniel/Pictures/Photos1/1/' + 'duolingo_' + data.split('/')[-1])
cc1 += 1
else:
pass
#print('not sure')
cc2 += 1
print(cc)
print(f'cc0: {cc0}')
print(f'cc1: {cc1}')
print(f'cc2: {cc2}')
Results after 3 runs
365
cc0: 152
cc1: 195
cc2: 18
365
cc0: 154
cc1: 186
cc2: 25
365
cc0: 136
cc1: 206
cc2: 23