convert image to list of rgb values [duplicate] - python-3.x
According to https://pillow.readthedocs.io/en/3.1.x/handbook/concepts.html#concept-modes,
What are the difference between them?
Can we convert from one to another?
What are the examples of image for both mode?
Normally, images are RGB, which means they have 3 channels, one for red, one for green and one for blue. That normally means that each pixel takes 3 bytes of storage, one for red, one for green and one for blue.
If you have a P mode image, that means it is palettised. That means there is a palette with up to 256 different colours in it, and instead of storing 3 bytes for R, G and B for each pixel, you store 1 byte which is the index into the palette. This confers both advantages and disadvantages. The advantage is that your image requires 1/3 of the space in memory and on disk. The disadvantage is that it can only represent 256 unique colours - so you may get banding or artefacts.
If you have an L mode image, that means it is a single channel image - normally interpreted as greyscale. The L means that is just stores the Luminance. It is very compact, but only stores a greyscale, not colour.
You can tell which mode your image has by looking at:
image.mode
If your image is palettised it will be P, or PA if palettised with an alpha channel as well. If your image is greyscale it will be L, or LA if greyscale with alpha channel.
You convert between them using the convert(mode) function, e.g. to go to RGB mode, use:
image.convert('RGB')
I used the word "normally" quite a lot! Why? Because you can do abnormal things!
You can store a grey-looking image in an RGB format. All you do, is make the red component equal to the green component equal to the blue component (R=G=B) and it will appear grey but be stored in an inefficient RGB format that takes 3x the space it might otherwise need to.
You can store a grey-looking image in a P format, you just make sure all the palette entries have the R=G=B.
Here's the kicker... if you want and expect an RGB image, you should just convert to RGB on opening:
im = Image.open("image.jpg").convert('RGB')
that way you will never have problems with GIF files (which are always palettised) nor with PNG files which can be palettised and can be greyscale or RGB. You will not normally get problems with JPEG images because they are pretty much always RGB anyway.
Here's an example to demonstrate. Start with this red-blue gradient image:
Let's use IPython to look at in RGB space. First, look at the Red channel:
In [21]: im = Image.open('a.png').convert('RGB')
In [22]: np.array(im.getchannel(0))
Out[22]:
array([[255, 255, 255, ..., 255, 255, 255],
[255, 255, 255, ..., 255, 255, 255],
[254, 254, 254, ..., 254, 254, 254],
...,
[ 1, 1, 1, ..., 1, 1, 1],
[ 0, 0, 0, ..., 0, 0, 0],
[ 0, 0, 0, ..., 0, 0, 0]], dtype=uint8)
Notice it has 255 at the top because it is red, and 0 at the bottom because there is no red there.
Now let's look at the Green channel, it is 0 everywhere because there is no green.
In [23]: np.array(im.getchannel(1))
Out[23]:
array([[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]], dtype=uint8)
And finally, let's look at the Blue channel. It is 0 at the top where the image is pure Red and 255 at the bottom where the image is pure Blue.
In [24]: np.array(im.getchannel(2))
Out[24]:
array([[ 0, 0, 0, ..., 0, 0, 0],
[ 0, 0, 0, ..., 0, 0, 0],
[ 1, 1, 1, ..., 1, 1, 1],
...,
[254, 254, 254, ..., 254, 254, 254],
[255, 255, 255, ..., 255, 255, 255],
[255, 255, 255, ..., 255, 255, 255]], dtype=uint8)
Now let's look at the same image in palette mode.
# Convert to palette mode
im = Image.open('a.png').convert('P')
# Extract the palette and reshape as 256 entries of 3 RGB bytes each
In [27]: np.array(im.getpalette()).reshape(256,3)
Out[27]:
array([[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 51, 0, 0],
[102, 0, 0],
[153, 0, 0],
[204, 0, 0],
[255, 0, 0], <--- entry 15 = rgb(255,0,0) = Red
[ 0, 51, 0],
[ 51, 51, 0],
[102, 51, 0],
[153, 51, 0],
[204, 51, 0],
[255, 51, 0],
[ 0, 102, 0],
[ 51, 102, 0],
[102, 102, 0],
[153, 102, 0],
[204, 102, 0],
[255, 102, 0],
[ 0, 153, 0],
[ 51, 153, 0],
[102, 153, 0],
[153, 153, 0],
[204, 153, 0],
[255, 153, 0],
[ 0, 204, 0],
[ 51, 204, 0],
[102, 204, 0],
[153, 204, 0],
[204, 204, 0],
[255, 204, 0],
[ 0, 255, 0],
[ 51, 255, 0],
[102, 255, 0],
[153, 255, 0],
[204, 255, 0],
[255, 255, 0],
...
... up to 256 entries
Now get the indices into the palette:
In [28]: np.array(im.getchannel(0))
Out[28]:
array([[ 15, 15, 15, ..., 15, 15, 15],
[ 15, 15, 15, ..., 15, 15, 15],
[ 15, 15, 15, ..., 15, 15, 15],
...,
[190, 190, 190, ..., 190, 190, 190],
[190, 190, 190, ..., 190, 190, 190],
[190, 190, 190, ..., 190, 190, 190]], dtype=uint8)
Now you can see that the top row of the image has palette index 15, which, if you look it up in the preceding palette, you will see is Red.
Now let's look at the same image in L mode - remember L means "Luminance" which is just a fancy way of saying "brightness" on a scale of black to white, i.e. greyscale :
# Open into greyscale, or L mode
In [1]: im = Image.open('a.png').convert('L')
# Dump the pixels
In [2]: np.array(im.getchannel(0))
Out[2]:
array([[76, 76, 76, ..., 76, 76, 76],
[76, 76, 76, ..., 76, 76, 76],
[76, 76, 76, ..., 76, 76, 76],
...,
[29, 29, 29, ..., 29, 29, 29],
[29, 29, 29, ..., 29, 29, 29],
[29, 29, 29, ..., 29, 29, 29]], dtype=uint8)
So, now the top row of the image is 76 and the bottom row is 29. What are those? Well, the formula for converting RGB to L is:
L = R * 299/1000 + G * 587/1000 + B * 114/1000
So, in the top row, R=255, G=0, B=0, so the Luminance has become:
L = 255 * 299/1000 + 0 + 0
L = 76
And on the bottom row, R=0, G=0, B=255, so the Luminance has become:
L = 0 + 0 + 255 * 114/1000
L = 29
Keywords: Python, PIL, Pillow, palette, image processing, prime.
"L" mode maps to black and white pixels (and in between). "P" mode maps with a color palette. You can convert image to one of these modes.
from PIL import Image
im = Image.open("im.jpg")
im_l = im.convert('L')
im_p = im.convert('P')
im.show()
im_l.show()
im_p.show()
The "L" mode represents grayscale here.... So it can hold any of 256 shades of Gray (includes black and white as Gray shades).
The "P" mode can hold 256 different colors like red,blue, green etc....
Convertion from one another, if you mean converting images from grayscale to color or vice versa.... Yes it is possible....
Examples: 8 bit black and white image ( technically Gray scale image)are "L" and any 8 bit color images are "P" mode..
Related
2D Array Slice Assignment in loop not working as expected?
Below is the code code for slice assignment For loop is used to assign lists to slice of list from main 2d list But the output is not as expected. qu = [[1, 2, 100], [2, 5, 100], [3, 4, 100]] n=5 a = [[0]*n]*len(qu) for i in range(len(qu)): p=qu[i][0]-1 q=qu[i][1] a[i][p:q]=[qu[i][2]]*(q-p) print(a) Output --- [[100, 100, 0, 0, 0], [100, 100, 0, 0, 0], [100, 100, 0, 0, 0]] [[100, 100, 100, 100, 100], [100, 100, 100, 100, 100], [100, 100, 100, 100, 100]] [[100, 100, 100, 100, 100], [100, 100, 100, 100, 100], [100, 100, 100, 100, 100]] Expected Output -- [[100, 100, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0]] [[100, 100, 0, 0, 0], [0, 100, 100, 100, 100], [0, 0, 0, 0, 0]] [[100, 100, 0, 0, 0], [0, 100, 100, 100, 100], [0, 0, 100, 100, 0]] Blockquote
Problems with creating a stacked bar plot in Matplotlib
I have 5 lists with 7 items each: A=[0, 2, 0, 154018, 0, 0, 0] B=[0, 22, 0, 153882, 0, 0, 0] C=[0, 17, 0, 152901, 0, 0, 0] D=[0, 14, 4, 154302, 0, 0, 0] E=[0, 18, 8, 155052, 0, 0, 0] I wold like to get a stacked bar plot according to each stations. I tried with the below code but getting error: df = pd.DataFrame([['Stn1', A], ['Stn2', B], ['Stn3', C], ['Stn4', D],['Stn5', E]], columns=['RU_26', 'RU_52', 'RU_106', 'RU_242', 'RU_484','RU_996','RU_2x996']) Could someone help with this?
How to get important words using LGBM feature importance and Tfidf vectorizer?
I am working a Kaggle dataset that predicts a price of an item using its description and other attributes. Here is the link to the competition. As part of an experiment, I am currently, only using an item's description to predict its price. The description is free text and I use sklearn's Tfidf's vectorizer with a bi-gram and max features set to 60000 as input to a lightGBM model. After training, I would like to know the most influential tokens for predicting the price. I assumed lightGBM's feature_importance method will be able to give me this. This will return a 60000 dim numpy array, whose index I can use to retrieve the token from the Tfidf's vectorizer's vocab dictionary. Here is the code: vectorizer = TfidfVectorizer(ngram_range=(1,2), max_features=60000) x_train = vectorizer.fit_transform(train_df['text'].values.astype('U')) x_valid = vectorizer.transform(valid_df['text'].values.astype('U')) idx2tok = {v: k for k, v in vectorizer.vocabulary_.items()} features = [f'token_{i}' for i in range(len(vectorizer.vocabulary_))] get_tok = lambda x, idxmap: idxmap[int(x[6:])] lgb_train = lgb.Dataset(x_train, y_train) lgb_valid = lgb.Dataset(x_valid, y_valid, reference=lgb_train) gbm = lgb.train(lgb_params, lgb_train, num_boost_round=10, valid_sets=[lgb_train, lgb_valid], early_stopping_rounds=10, verbose_eval=True) The model trains, however, after training when I call gbm.feature_importance(), I get a sparse array of integers, that really doesn't make sense to me: fi = gbm.feature_importance() fi[:100] array([ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32) np.unique(fi) array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 33, 34, 38, 45], dtype=int32) I'm not sure how to interpret this. I thought that earlier indices of the feature importance array will have higher value and thus tokens corresponding to that index in the vectorizer's vocab will be more important/influential than other tokens. Is this assumption wrong? How do I get the most influential/important terms that determines the model outcome? Any help is appreciated. Thanks.
In-place sorting of csc_matrix columns
I want to be able to sort columns of a scipy sparse matrix. The scipy documentation is fairly terse, and I can't see much concerning modification of the matrix. On SO I found this post, but the answer given returns a list The code that I want to write is s = rand(4, 4, density=0.25, format='csc') _,colSize = s.get_shape() for j in range(0,colSize): s.setcol(j, sorted(s.getcol(j), key=attrgetter('data'), reverse=True)) Except there is no setcol and sorted doesn't return the same type as getcol. As an example of what I'd like to get, if I have in input <class 'scipy.sparse.csc.csc_matrix'> [[ 0. 0.33201655 0. 0. ] [ 0. 0. 0. 0. ] [ 0. 0.81332962 0. 0.50794041] [ 0. 0.41478979 0. 0. ]] then the output that I want is [[ 0. 0.81332962 0. 0.50794041] [ 0. 0.414789790. 0. 0. ] [ 0. 0.332016550. 0. 0. ] [ 0. 0. 0. 0. ]] (It doesn't have to be a csc matrix, I assumed that this would be better for column manipulations)
Here's a short function that sorts the columns in descending order in-place: import numpy as np def sort_csc_cols(m): """ Sort the columns of m in descending order. m must be a csc_matrix whose nonzero values are all positive. m is modified in-place. """ seq = np.arange(m.shape[0]) for k in range(m.indptr.size - 1): start, end = m.indptr[k:k + 2] m.data[start:end][::-1].sort() m.indices[start:end] = seq[:end - start] For example, s is a csc_matrix: In [47]: s Out[47]: <8x12 sparse matrix of type '<class 'numpy.int64'>' with 19 stored elements in Compressed Sparse Column format> In [48]: s.A Out[48]: array([[ 0, 2, 0, 0, 7, 0, 0, 48, 0, 0, 0, 0], [ 0, 0, 82, 0, 0, 38, 67, 17, 9, 0, 0, 0], [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 47, 0], [ 0, 0, 0, 0, 0, 0, 99, 0, 0, 0, 0, 0], [ 0, 0, 0, 0, 0, 0, 0, 83, 0, 0, 0, 9], [ 0, 0, 0, 0, 0, 0, 85, 94, 0, 55, 68, 0], [ 0, 0, 0, 0, 0, 0, 22, 0, 0, 0, 71, 0], [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]) In [49]: sort_csc_cols(s) In [50]: s.A Out[50]: array([[ 0, 2, 82, 0, 7, 38, 99, 94, 9, 55, 71, 9], [ 0, 0, 0, 0, 0, 0, 85, 83, 0, 0, 68, 0], [ 0, 0, 0, 0, 0, 0, 67, 48, 0, 0, 47, 0], [ 0, 0, 0, 0, 0, 0, 22, 17, 0, 0, 0, 0], [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])
Using dijkstra_path function in networkx library
I'm using networkx library to find shortest path between two nodes using dijkstra algo as follows import networkx as nx A = [[0, 100, 0, 0 , 40, 0], [100, 0, 20, 0, 0, 70], [0, 20, 0, 80, 50, 0], [0, 0, 80, 0, 0, 30], [40, 0, 50, 0, 0, 60], [0, 70, 0, 30, 60, 0]]; print(nx.dijkstra_path(A, 0, 4)) In the above code I'm using matrix directly, But library requires graph to be created as follows G = nx.Graph() G = nx.add_node(<node>) G.add_edge(<node 1>, <node 2>) It is very time consuming to create matrix by using above commands. Is there any way to give input as weighted matrix to the dijkstra_path function.
First you need to convert your adjacency matrix to a numpy matrix with np.array. Then you can simply create your graph with from_numpy_matrix. import networkx as nx import numpy as np A = [[0, 100, 0, 0 , 40, 0], [100, 0, 20, 0, 0, 70], [0, 20, 0, 80, 50, 0], [0, 0, 80, 0, 0, 30], [40, 0, 50, 0, 0, 60], [0, 70, 0, 30, 60, 0]] a = np.array(A) G = nx.from_numpy_matrix(a) print(nx.dijkstra_path(G, 0, 4)) Output: [0, 4] Side note: you can check the graph edges with the following code. for edge in G.edges(data=True): print(edge) Output: (0, 1, {'weight': 100}) (0, 4, {'weight': 40}) (1, 2, {'weight': 20}) (1, 5, {'weight': 70}) (2, 3, {'weight': 80}) (2, 4, {'weight': 50}) (3, 5, {'weight': 30}) (4, 5, {'weight': 60})