I have trained a model on a deep fashion dataset and now I am doing inference. The thing is that I am not getting the full result like I got when I tested the model after I trained it. Here are the pkl and pth files
This is my inference script
import torch
import torchvision.models as models
from torchvision import transforms
from torch.autograd import Variable
from PIL import Image
import torch.nn as nn
MODEL_PATH = "/content/deepfashion-dataset/models/atr-recognition-stage-2-resnet34.pkl"
DATA_PATH = "/content/deepfashion-dataset/"
CLASSES_PATH = "/content/deepfashion-dataset/clothes_categories/attribute-classes.txt"
class ClassificationModel():
def __init__(self):
return
def load(self, model_path, labels_path, eval=False):
self.model = torch.load(model_path)
self.model = nn.Sequential(self.model)
self.labels = open(labels_path, 'r').read().splitlines()
if eval:
print(model.eval())
return
def predict(self, image_path):
device = torch.device("cpu")
img = Image.open(image_path)
test_transforms = transforms.Compose([transforms.Resize(224),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406],
[0.229, 0.224, 0.225])
])
image_tensor = test_transforms(img).float()
image_tensor = image_tensor.unsqueeze_(0)
inp = Variable(image_tensor)
inp = inp.to(device)
output = self.model(inp)
index = output.data.cpu().numpy().argmax()
return self.labels[index]
learner = ClassificationModel()
learner.load(MODEL_PATH, CLASSES_PATH)
print(learner.predict(DATA_PATH+"img-lk/IMG_20210530_152109_062.jpg"))
The output it gives for this image is:
leather
While when I tested it after training the model like this:
def predict_attribute(model, path, display_img=True):
predicted = model.predict(path)
if display_img:
size = 244,244
img=Image.open(path)
img.thumbnail(size,Image.ANTIALIAS)
display(img)
return predicted[0]
image_path = PATH + 'img-lk/IMG_20210530_152109_062.jpg'
predict_attribute(learn, image_path)
Output was:
(#2) ['faux-leather','leather']
It gave both the output attributes that time. Also my attributes are saved in a text file.
Related
According to the formula of BatchNorm, the output should have negative values, but only positive values and 0 after BatchNorm2d.
I use hooks to get the output of the "layer4.1.bn2" in Resnet.
import torch
import torchvision
from PIL import Image
from torchvision import transforms
class HookTool:
def __init__(self):
self.fea = None
def hook_fun(self, module, fea_in, fea_out):
self.fea = fea_out
def get_feas_by_hook(model):
print('begin hooks...')
fea_hooks = []
for n, m in model.named_modules():
if isinstance(m, torch.nn.BatchNorm2d) and n == 'layer4.1.bn2':
cur_hook = HookTool()
m.register_forward_hook(cur_hook.hook_fun)
fea_hooks.append(cur_hook)
return fea_hooks
model = torchvision.models.resnet18(pretrained=True)
transform = transforms.Compose([
# transforms.CenterCrop(224),
transforms.Resize((224, 224)),
transforms.ToTensor(),
# transforms.Normalize(mean=(0.5, 0.5, 0.5), std=(0.5, 0.5, 0.5))
])
fea_hooks = get_feas_by_hook(model)
img = Image.open("ILSVRC2012_val_00002138.JPEG")
input = transform(img)
input = input.reshape(1, 3, 224, 224)
print(input.shape)
output = model(input)
print('The number of hooks is:', len(fea_hooks))
print('The shape of the first Conv2D feature is:', fea_hooks[0].fea.shape)
When I output the "fea_hooks", there are no negative numbers.
And why?
I'm trying to one time run my model and see if it's working. I've searched the error and suggested answers was to instantiate the model once. I already did that. However, even after instantiating the model. it gives an error once requesting access to model parametes. What's the problem?
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import DataLoader, TensorDataset
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
from IPython import display
display.set_matplotlib_formats('svg')
data = np.array([[1,1]])
label = np.array([2])
for i in range(-50,51,1):
data = np.append(data, [[i,i]], axis=0)
label = np.append(label, [i+i])
# conver to tensor
T_data = torch.tensor(data).float()
T_label = torch.tensor(label).long()
# split data
train_data, test_data, train_label, test_label = train_test_split(T_data, T_label, test_size= .2)
# convert into Pytorch dataset
train_data = TensorDataset(train_data, train_label)
test_data = TensorDataset(test_data, test_label)
# translate into dataloader
batchsize = 32
train_loader = DataLoader(train_data, batch_size= batchsize, shuffle =True, drop_last=True)
test_loader = DataLoader(test_data, batch_size=test_data.tensors[0].shape[0])
class AddNN(nn.Module):
def __init__(self):
super().__init__()
self.input = nn.Linear(2,16)
## hidden layer
self.fc1 = nn.Linear(16,32)
self.fc2 = nn.Linear(32,1)
#output layer
self.output = nn.Linear(1,1)
# forward pass
def forward(self, x):
x = F.relu(self.input(x))
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
return self.output(x)
net =AddNN()
lossfun = nn.MSELoss()
optimizer = torch.optim.Adam(AddNN.parameters(), lr = .05)
It returns TypeError: parameters() missing 1 required positional argument: 'self'
What it means to have missing a required positional argument here??
After model instantiation, you should use that model object instead of the model class inside Adam and with .parameters()
net = AddNN()
lossfun = nn.MSELoss()
optimizer = torch.optim.Adam(net.parameters(), lr = .05)
Please help me out, I am very confused. The dataset contains three folders train, nuisances, and testing set. The train folder contains 10 classes like aeroplane, train, etc. However, the nuisances folder consists of 5 classes like nuisances = ['shape', 'pose', 'texture', 'context', 'weather', 'occlusion'] and each class contains 10 classes same as the training dataset.
Dataset Structure
- dataset
- train folder
- images
- aeroplane
- train
- etc
- labels.csv
- nuisances folder
-context
- Images
- labels.csv
-occlusion
- Images
- labels.csv
- test folder
- images
- labels.csv
When testing, the test script generates a single .csv file containing all classes img_name, preds.
Code
from torch.utils.data import Dataset, DataLoader
import pandas as pd
from torchvision import transforms
from PIL import Image
import torch
import torch.nn as nn
from glob import glob
from pathlib import PurePath
import numpy as np
import timm
import torchvision
import time
img_list = glob('/media/cvpr/CM_22/ROBINv1.1-cls-pose/iid_test/Images/*.jpg')
name_list = [
'aeroplane',
'bicycle',
'boat',
'bus',
'car',
'chair',
'diningtable',
'motorbike',
'sofa',
'train'
]
# conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 -c pytorch
class PoseData(Dataset):
def __init__(self, transforms) -> None:
"""
the data folder should look like
- datafolder
- Images
- labels.csv
"""
super().__init__()
self.img_list = glob('/media/cvpr/CM_22/ROBINv1.1-cls-pose/iid_test/Images/*.jpg')
self.img_list = sorted(self.img_list, key=lambda x: PurePath(x).parts[-1][:-4])
self.trs = transforms
def __len__(self):
return len(self.img_list)
def __getitem__(self, index):
image_dir = self.img_list[index]
image_name = PurePath(image_dir).parts[-1]
image = Image.open(image_dir)
image = self.trs(image)
return image, image_name
if __name__ == "__main__":
normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
tfs = transforms.Compose([
transforms.Resize((224, 224)),
transforms.ToTensor(),
normalize,
])
model = timm.models.resnet152(pretrained=True, num_classes=10)
#model = torch.nn.DataParallel(model)
model.load_state_dict(torch.load('resnet152_best.pth.tar')['state_dict'], strict=False)
model = model.cuda()
model.eval()
dataset = PoseData(tfs)
loader = DataLoader(dataset, batch_size=128, shuffle=False, drop_last=False, num_workers=4)
image_dir = []
preds = []
for image, pth in loader:
image_dir.append(list(pth))
image = image.cuda()
with torch.no_grad():
model.eval()
pred = model(image)
#pred = torch.softmax(pred[:, :10], dim=1)
pred = torch.argmax(pred[:, :10], dim=1)
p = []
for i in range(pred.size(0)):
p.append(name_list[pred[i].item()])
p = np.array(p)
preds.append(p)
print(len(np.concatenate(preds)))
image_dir = np.array(sum(image_dir, []))
preds = np.concatenate(preds)
csv = {'imgs': np.array(image_dir), 'pred': np.array(preds),
}
csv = pd.DataFrame(csv)
print(csv)
csv.to_csv('evaluation/cls_ref/res/iid.csv', index=False)
The iid.csv is completely fine but I also require the separate .csv file for nuisances = ['shape', 'pose', 'texture', 'context', 'weather', 'occlusion'] so that I can evaluate accuracy on each nuisances classes. I have no idea how to build logic for this task
I am trying to fine tune GPT2, with Huggingface's trainer class.
from datasets import load_dataset
import torch
from torch.utils.data import Dataset, DataLoader
from transformers import GPT2TokenizerFast, GPT2LMHeadModel, Trainer, TrainingArguments
class torchDataset(Dataset):
def __init__(self, encodings):
self.encodings = encodings
self.len = len(encodings)
def __getitem__(self, index):
item = {torch.tensor(val[index]) for key, val in self.encodings.items()}
return item
def __len__(self):
return self.len
def print(self):
print(self.encodings)
# HYPER PARAMETERS
EPOCHS = 5
BATCH_SIZE = 2
WARMUP_STEPS = 5000
LEARNING_RATE = 1e-3
DECAY = 0
# Model ids and loading dataset
model_id = 'gpt2' # small model
# model_id = 'gpt2-medium' # medium model
# model_id = 'gpt2-large' # large model
dataset = load_dataset('wikitext', 'wikitext-2-v1') # first dataset
# dataset = load_dataset('m-newhauser/senator-tweets') # second dataset
# dataset = load_dataset('IsaacRodgz/Fake-news-latam-omdena') # third dataset
print('Loaded dataset')
# Dividing dataset into predefined splits
train_dataset = dataset['train']['text']
validation_dataset = dataset['validation']['text']
test_dataset = dataset['test']['text']
print('Divided dataset')
# loading tokenizer
tokenizer = GPT2TokenizerFast.from_pretrained(model_id,
# bos_token='<|startoftext|>', eos_token='<|endoftext|>',
pad_token='<|pad|>'
)
print('tokenizer max length:', tokenizer.model_max_length)
train_encoding = tokenizer(train_dataset, padding=True, truncation=True, max_length=1024, return_tensors='pt')
eval_encoding = tokenizer(validation_dataset, padding=True, truncation=True, max_length=1024, return_tensors='pt')
test_encoding = tokenizer(test_dataset, padding=True, truncation=True, max_length=1024, return_tensors='pt')
print('Converted to torch dataset')
torch_dataset_train = torchDataset(train_encoding)
torch_dataset_eval = torchDataset(eval_encoding)
torch_dataset_test = torchDataset(test_encoding)
# Setup training hyperparameters
training_args = TrainingArguments(
output_dir='/model_dump/',
num_train_epochs=EPOCHS,
warmup_steps=WARMUP_STEPS,
learning_rate=LEARNING_RATE,
weight_decay=DECAY
)
model = GPT2LMHeadModel.from_pretrained(model_id)
model.resize_token_embeddings(len(tokenizer))
trainer = Trainer(
model=model,
args=training_args,
train_dataset=train_encoding,
eval_dataset=eval_encoding
)
trainer.train()
# model.save_pretrained('/model_dump/')
But with this code I get this error
The batch received was empty, your model won't be able to train on it. Double-check that your training dataset contains keys expected by the model: input_ids,past_key_values,attention_mask,token_type_ids,position_ids,head_mask,inputs_embeds,encoder_hidden_states,encoder_attention_mask,labels,use_cache,output_attentions,output_hidden_states,return_dict,labels,label,label_ids.
When I use the variables torch_dataset_train and torch_dataset_eval in Trainer's arguments, the error I get is:
TypeError: vars() argument must have __dict__ attribute
This typeError is the same I get if as dataset I use the WikiText2 from torchtext.
How can I fix this issue?
Using this mnist image classification model :
%reset -f
import torch
import torch.nn as nn
import torchvision
import torchvision.transforms as transforms
import torch
import torch.nn as nn
import torchvision
import torchvision.transforms as transforms
import torch.utils.data as data_utils
import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_moons
from matplotlib import pyplot
from pandas import DataFrame
import torchvision.datasets as dset
import os
import torch.nn.functional as F
import time
import random
import pickle
from sklearn.metrics import confusion_matrix
import pandas as pd
import sklearn
trans = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5,), (1.0,))])
root = './data'
if not os.path.exists(root):
os.mkdir(root)
train_set = dset.MNIST(root=root, train=True, transform=trans, download=True)
test_set = dset.MNIST(root=root, train=False, transform=trans, download=True)
batch_size = 64
train_loader = torch.utils.data.DataLoader(
dataset=train_set,
batch_size=batch_size,
shuffle=True)
test_loader = torch.utils.data.DataLoader(
dataset=test_set,
batch_size=batch_size,
shuffle=True)
class NeuralNet(nn.Module):
def __init__(self):
super(NeuralNet, self).__init__()
self.fc1 = nn.Linear(28*28, 500)
self.fc2 = nn.Linear(500, 256)
self.fc3 = nn.Linear(256, 2)
def forward(self, x):
x = x.view(-1, 28*28)
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
x = self.fc3(x)
return x
num_epochs = 2
random_sample_size = 200
values_0_or_1 = [t for t in train_set if (int(t[1]) == 0 or int(t[1]) == 1)]
values_0_or_1_testset = [t for t in test_set if (int(t[1]) == 0 or int(t[1]) == 1)]
print(len(values_0_or_1))
print(len(values_0_or_1_testset))
train_loader_subset = torch.utils.data.DataLoader(
dataset=values_0_or_1,
batch_size=batch_size,
shuffle=True)
test_loader_subset = torch.utils.data.DataLoader(
dataset=values_0_or_1_testset,
batch_size=batch_size,
shuffle=False)
train_loader = train_loader_subset
# Hyper-parameters
input_size = 100
hidden_size = 100
num_classes = 2
# learning_rate = 0.00001
learning_rate = .0001
# Device configuration
device = 'cpu'
print_progress_every_n_epochs = 1
model = NeuralNet().to(device)
# Loss and optimizer
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
N = len(train_loader)
# Train the model
total_step = len(train_loader)
most_recent_prediction = []
test_actual_predicted_dict = {}
rm = random.sample(list(values_0_or_1), random_sample_size)
train_loader_subset = data_utils.DataLoader(rm, batch_size=4)
for epoch in range(num_epochs):
for i, (images, labels) in enumerate(train_loader_subset):
# Move tensors to the configured device
images = images.reshape(-1, 2).to(device)
labels = labels.to(device)
# Forward pass
outputs = model(images)
loss = criterion(outputs, labels)
# Backward and optimize
optimizer.zero_grad()
loss.backward()
optimizer.step()
if (epoch) % print_progress_every_n_epochs == 0:
print ('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}'.format(epoch+1, num_epochs, i+1, total_step, loss.item()))
predicted_test = []
model.eval() # eval mode (batchnorm uses moving mean/variance instead of mini-batch mean/variance)
probs_l = []
predicted_values = []
actual_values = []
labels_l = []
with torch.no_grad():
for images, labels in test_loader_subset:
images = images.to(device)
labels = labels.to(device)
outputs = model(images)
_, predicted = torch.max(outputs.data, 1)
predicted_test.append(predicted.cpu().numpy())
sm = torch.nn.Softmax()
probabilities = sm(outputs)
probs_l.append(probabilities)
labels_l.append(labels.cpu().numpy())
predicted_values.append(np.concatenate(predicted_test).ravel())
actual_values.append(np.concatenate(labels_l).ravel())
if (epoch) % 1 == 0:
print('test accuracy : ', 100 * len((np.where(np.array(predicted_values[0])==(np.array(actual_values[0])))[0])) / len(actual_values[0]))
I'm to attempting to integrate 'Local Interpretable Model-Agnostic Explanations for machine learning classifiers' : https://marcotcr.github.io/lime/
It appears PyTorch support is not enabled as it is not mentioned in doc and following tutorial :
https://marcotcr.github.io/lime/tutorials/Tutorial%20-%20images.html
With my updated code for PyTorch :
from lime import lime_image
import time
explainer = lime_image.LimeImageExplainer()
explanation = explainer.explain_instance(images[0].reshape(28,28), model(images[0]), top_labels=5, hide_color=0, num_samples=1000)
Causes error :
/opt/conda/lib/python3.6/site-packages/skimage/color/colorconv.py in gray2rgb(image, alpha)
830 is_rgb = False
831 is_alpha = False
--> 832 dims = np.squeeze(image).ndim
833
834 if dims == 3:
AttributeError: 'Tensor' object has no attribute 'ndim'
So appears tensorflow object is expected here ?
How to integrate LIME with PyTorch image classification ?
Here's my solution:
Lime expects an image input of type numpy. This is why you get the attribute error and a solution would be to convert the image (from Tensor) to numpy before passing it to the explainer object. Another solution would be to select a specific image with the test_loader_subset and convert it with img = img.numpy().
Secondly, in order to make LIME work with pytorch (or any other framework), you'll need to specify a batch prediction function which outputs the prediction scores of each class for each image. The name of this function (here I've called it batch_predict) is then passed to explainer.explain_instance(img, batch_predict, ...). The batch_predict needs to loop through all images passed to it, convert them to Tensor, make a prediction and finally return the prediction score list (with numpy values). This is how I got it working.
Note also that the images need to have shape (... ,... ,3) or (... ,... ,1) in order to be properly segmented by the default segmentation algorithm. This means that you might have to use np.transpose(img, (...)). You may specify the segmentation algorithm as well if the results are poor.
Finally you'll need to display the LIME image mask on top of the original image. This snippet shows how this may be done:
from skimage.segmentation import mark_boundaries
temp, mask = explanation.get_image_and_mask(explanation.top_labels[0], positive_only=False, num_features=5, hide_rest=False)
img_boundry = mark_boundaries(temp, mask)
plt.imshow(img_boundry)
plt.show()
This notebook is a good reference:
https://github.com/marcotcr/lime/blob/master/doc/notebooks/Tutorial%20-%20images%20-%20Pytorch.ipynb