I have a trained CNN on CIFAR10 data, and I would like to see what the filters are doing. I know how to see the filters for a greyscale image, but am having trouble understanding how to get the output of my first convolutional layer and properly see it as an RGB image when the input is an RGB image. For reference, I know that for an image from the MNIST dataset
from tensorflow.keras.models import Model
import matplotlib.pyplot as plt
# model is trained CNN
# Creating model out of first layer of the CNN
fmap_model = Model(inputs=model.inputs, outputs=model.layers[0].output)
val = 0
img = x_train[samples[val]].reshape(1,28,28,1)
# Getting the filtered image for each of the (32) filters
feature_maps = fmap_model.predict(img)
plt.imshow(feature_maps[0,:,:,0], cmap='binary', vmin=0, vmax=1)
gives me a plot of the image under the first filter in the CNN. I would like to know if it is possible to do the same with RGB images and how I can adapt my code to do so.
I have tried a few different, things, but they all seem to be stuck on the fact that the shape of feature_maps[0,:,:,0] is (32,32) and not (32,32,3). Is there a way to get the output to be represented in the RGB channels instead of just one?
Related
I have a training data features in ndarray of shape (100, 400, 3) as it's 100 images of 20x20 with RGB channel and label in shape (100, ). Do I need to combine them into one dataset or how can I pass it to Pytorch dataLoader in order to iterate over image and label later?
What I've tried so far
#turn ndarray of features and labels into tensors
transform = transforms.Compose([transforms.ToPILImage(),
transforms.ToTensor()])
As #Shai mentioned, DataLoader requires the input to be the Dataset class or its subclass. One of the simplest subclasses is TensorDataset and you can convert it from ndarray.
import torch
import numpy as np
import torch.utils as utils
train_x = torch.Tensor(np.random.randn(100,400,3))
train_y = torch.Tensor(np.random.randint(0,2,100))
dataset = utils.data.TensorDataset(train_x, train_y)
dataloader = utils.data.DataLoader(dataset)
You can convert your data/label ndarrays to torch.tensor and use torch.utils.data.TensorDataset to create a dataset that iterates over your examples.
Once you have a dataset, you can wrap a DataLoader around it to be used for training.
I am working on image_classification problem(multi-class).
i am using resnet50 model( https://keras.io/applications/#classify-imagenet-classes-with-resnet50 ) along with pretrained db "imagenet" using keras
I am getting the the output labels for which the images i passed to the model.
But now,
i have image data and label data with me of my own dataset.
When i pass the images to the resnet50 model it gives back the imagenet labels that are already trained. Now, here, i want the output as my own labels which is already in dataset instead of getting imagenet labels.
How to to fine tune labels in resnet50 model with imagenet db in keras
I have tried the resnet50 model alone and it works fine. but, how to change the output to my own labels instead of imagenet pre-trained labels.
from keras.applications.resnet50 import ResNet50
from keras.preprocessing import image
from keras.applications.resnet50 import preprocess_input, decode_predictions
import numpy as np
import os
model = ResNet50(weights='imagenet')
path='/Users/resnet-sample/'
img_path=os.listdir(path)
count=0
for i in img_path:
img = image.load_img(path+i, target_size=(224, 224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
preds = model.predict(x)
print('Predicted:', decode_predictions(preds, top=1)[0], i)
count=count+1
print(preds)
example:
i have an elephant image in jpg format and label its as an 'elephant' in my dataset.
when i pass this image to resnet50 model which uses imagenet pre-trained db the output i received is 'African-Elephant'(imagenet-label).
So instead of getting imagenet label as output, i want to tune this as 'elephant' as label which is in my dataset.
So, not sure how to fine tune the last layers that gives output as my labels instead of imagenet labels.
Pelase help me on this.
Thanks,
Srknt73
The weights argument should be either None (random initialization), imagenet (pre-training on ImageNet), or the path to the weights file to be loaded. So you give the path to the file containing the labels of your dataset
After training model with ImageDataGenerator(1/255.), do I need to rescale image before predicting ?
I thought it is necessary but experiment result said NO.
I trained a Resnet50 model which has 37 class on top layer.
Model was trained with ImageDataGenerator like this.
datagen = ImageDataGenerator(rescale=1./255)
generator=datagen.flow_from_directory(
directory=os.path.join(os.getcwd(), data_folder),
target_size=(224,224),
batch_size=256,
classes=None,
class_mode='categorical')
history = model.fit_generator(generator, steps_per_epoch=generator.n / 256, epochs=10)
Accuracy achieved 98% after 10 epochs on my train dataset.
The problem is, when i tried to predict each image in TRAIN dataset, prediction was wrong ( result is 33 whatever input image was )
img_p = './data/pets/shiba_inu/shiba_inu_27.jpg'
img = cv2.imread(img_p, cv2.IMREAD_COLOR)
img = cv2.resize(img, (224,224))
img_arr = np.zeros((1,224,224,3))
img_arr[0, :, :, :] = img / 255.
pred = model.predict(img_arr)
yhat = np.argmax(pred, axis=1)
yhat is 5, but y is 33
When I replace this line
img_arr[0, :, :, :] = img / 255.
by this
img_arr[0, :, :, :] = img
yhat is exactly 33.
Someone might suggest to use predict_generator() instead of predict(), but I want to understand what I did wrong here.
I knew what's wrong here.
I'm using Imagenet pretrained model, which DO NOT rescale image by divide it to 255. I have to use resnet50.preprocess_input before train/test.
preprocess_input function can be found here.
https://github.com/keras-team/keras-applications/blob/master/keras_applications/imagenet_utils.py
You must do every preprocessing that you do on train data, on each data that you want to feed to your trained network. actually when, for example, you rescale train images and train a network, your network train to get a matrix with entries between 0 and 1 and find the proper category. so if after training phase, you feed an image without rescaling, you feed a matrix with entries between 0 and 255 to your trained network while your network did not learn how treat with such matrix.
If you are following pre-processing exactly same as at the time of training then, you might look at the part of your code where you are predicting class using yhat = np.argmax(pred, axis=1) my hunch is that there might be class mismatch in accordance to indexing, to check how your classes are indexed when you use flow_from_directory use class_map = generator.class_indices this will return you a dictionary which will show you how your classes are mapped against index.
Note: The reason I state this because I've faced similar problem, using Keras flow_from_directory doesn't sort classes and hence it's quite possible that your prediction class 1 lies on the index 10 while np.argmax will return you class 1'.
For semantic segmentations, you generally end up with the last layer being something like
output = Conv2D(num_classes, (1, 1), activation='softmax')
My question is, how do I prepare the labels for this? For example, if I have 10 classes to identify, each with a different colour. For each label image, do I need to apply masking for one particular colour, turn this into grayscale image so that I can compare with 1 filter from the model output? Or is there a way to pass one full RGB picture in as the label?
The output of your network will be an image with 10 channels, where each pixel will consist of a vector of probabilities that sum to one (due to the softmax). Example: [0.1,0.1,0.1,0.05,0.05,0.1,0.1,0.1,0.1,0.2]. You want your labels images to be in the same shape: an image with 10 channels, and each pixel is a binary vector with a 1 at the index of the class and 0 elsewhere. Your segmentation loss function is then the pixel-wise crossentropy.
For implementation: the softmax in keras has an axis parameter: https://keras.io/activations/#softmax
np_utils.to_categorical(labels, num_classes)
When labels are (row,col), the output shape will be: (row, col, num_classes)
example:
https://github.com/naomifridman/Unet_Brain_tumor_segmentation
How to use ImageDataGenerator and predict_generator on a single JPEG file in Keras?
I am having a single jpeg and i want to predict the probability using model trained using model.fita-generator function.
If you just have a single .jpeg, you don't need to use the ImageDataGenerator. In the code below I'm assuming you trained your model with RGB images sized 150px x 150px.
img = image.load_img(img_path, target_size=(150, 150))
img_tensor = image.img_to_array(img)
img_tensor = np.expand_dims(img_tensor, axis=0)
img_tensor /= 255.
model.predict(img_tensor)
For more info, check out Francois Chollet's excellent Ipython Notebooks. Specifically, Line (In [2]) of https://github.com/fchollet/deep-learning-with-python-notebooks/blob/master/5.4-visualizing-what-convnets-learn.ipynb
In this section, he looks at the intermediate activation layers for an image that wasn't in his train_generator. He loads in a model he created in another Ipython notebook: https://github.com/fchollet/deep-learning-with-python-notebooks/blob/master/5.2-using-convnets-with-small-datasets.ipynb