There are several good tutorials on how to visualize Filters and Feature Maps for 2D CNN (images) [1] [2]. However, I am yet to find an example on how the Filters and Feature Maps could be visualized/illustrated for 1D CNN (text).
Current thinking is to:
retrieve CNN weights and try to represent them as characters n-gram (equivalent of the 2D 8x8 kernels)
feed a text sample thru each model component and layer (in my case the model has 5 CNN modules) and try to represent them as a string (equivalent of 2D image transformation)
Any recommendation or advice is appreciated.
[1] https://machinelearningmastery.com/how-to-visualize-filters-and-feature-maps-in-convolutional-neural-networks/
[2] https://discuss.pytorch.org/t/visualize-feature-map/29597
Related
I am planning to use the wavelet transform to extract textural features from images for classification purpose. However, I am not sure about whether using wavelet transform is good choice and which type of wavelet should I choose?
This is one example: the scattering transform. The difference from a regular CNN is that the filters don't emerge from training the network, but are based on a 3D basis with desirable properties. The case above is not a 'deep' network either: the wavelet 'footprints' are fed to an SVM.
The convolution filters learned in the first layers of image classification networks usually look like edge detectors, like wavelet filters. One way to use wavelets in CNN is to have them in the first layers and freeze them: don't change these layers during training.
I have the following problem:
Input: a set of 6 images
Output: a probability for each image determining whether the image is the correct one out of the 6 images
I know how to create a CNN with keras, but not how to have multiple images as an input.
How would one solve this problem?
One way I can think of is to use a pre-trained model (VGG16 etc.) and extract out the vectors from some intermediate layer, then concat 6 vectors together then feed it into a neural network (or some other classification model) and train it as a multiclass classification task.
You can also use an Autoencoder and take the anomaly detection approach.
I have searched the code that uses list or array input data for training DQN code. But I have could not find any code.
Currently, I reference the reinforcement learning tutorial(DQN) of Pytorch.
However, this code uses image input data.
I want to know how to change the image input data to list or array input data.
(I need help to resolve my research that uses list input data. List input data shape is 1 by 9. )
In PyTorch, we deal with tensors. Images, text, even sounds can be transformed to tensors and then PyTorch models can learn on the data.
In PyTorch image classifier examples, you often see something like this, to transform images to tensors:
train_transform = transforms.Compose([
transforms.Resize(x),
...
transforms.ToTensor()
])
If your input is a numpy array x, you can convert it to a tensor like this:
torch.from_numpy(x)
You also have to pay attention to tensor dimensions, your input data needs to match what the model expects in the first layer.
My question is simple. I want to visualize what filters are used in ConvNet in deep layers to extract the features predicting the final model. By visualize I mean to save it in .png format like filters shown in final layer of https://s3-ap-south-1.amazonaws.com/av-blog-media/wp-content/uploads/2018/03/cnn_filters.png , we can actually see a car in final layer filters
I can visualise the filters of first convolutional layer by help provided from my own question Visualising Keras CNN final trained filters at each layer but that shows only to visualise for first layer. First layer filters look like some random coloured 3x3 pixel images. But I want to see the final layer filters like the car filter in the first link.
First layer filters look like some random coloured 3x3 pixel images. But I want to see the final layer filters like the car filter in the first link.
Even the article of car filter https://www.analyticsvidhya.com/blog/2018/03/essentials-of-deep-learning-visualizing-convolutional-neural-networks/ has code for only first layer
The Python library keras-vis is a great tool for visualizing CNNs. It can generate conv filter visualizations, dense layer visualizations, and attention maps. The latest release is quite old (and a little bit buggy), so I recommend installing from master:
pip install git+https://github.com/raghakot/keras-vis.git
You can adress the weights of the different layers by:
w = model.layers[i].get_weights()[0][:,:,:,:]
where i is the number of your layer.
In case of the picture in the link I am not sure if it is actually the weights or maybe the activation map which is shown. You could get that one by:
from keras import backend as K
get_output = K.function([model.layers[0].input],[cnn.layers[i].output])
output_normal = get_output([X])[0][m]
where m is the number of a certain image in X as input.
I tried to process the tweets dataset using CNN in Theano. Different from images, the lenght of different tweets (corresponding to the image shape) is variable. So the shape of each tweet is different. However, in Theano, the convolution need that the shape information are constant values. So my question is that is there some way to make the image_shape dynamic?
Kalchbrenner et. al (2015) implemented an CNN that accepts dynamic length input and pools them into k elements. If there are less than k elements to begin with, the remaining are zero-padded. Their experiments with sentence classification show that such networks successfully represent grammatical structures.
For details check out:
the paper (http://arxiv.org/pdf/1404.2188v1.pdf)
Matlab code (link on page 2 of the paper)
suggestion for DCNNs for Theano/Keras (https://github.com/fchollet/keras/issues/373)
Convolutional neural networks are really better suited to processing images.
For processing tweets, you might want to read about recursive neural networks.
http://nlp.stanford.edu/~socherr/EMNLP2013_RNTN.pdf