Related
I am trying to crop ROI in an image. I have the bounding box coordinates, but I don't understand how to use it to extract sub-array from the original image
Here's a snippet of my code:
`for j, box in enumerate(draw_boxes):
cv2.rectangle(orig_image,
(int(box[0]), int(box[1])),
(int(box[2]), int(box[3])),
(0, 0, 255), 2)
cv2.putText(orig_image, pred_classes[j],
(int(box[0]), int(box[1]-5)),
cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 255, 0),
2, lineType=cv2.LINE_AA)
print(box[0], box[1], box[2], box[3])
cropped = orig_image[int(box[0]): int(box[1]), int(box[2]): int(box[3])]
cv2.imwrite(f"../test_predictions/{image_name}.jpg", orig_image)
cv2.imshow('Cropped', cropped)
cv2.imshow('Prediction', orig_image)
cv2.waitKey()`
I encoded and decoded a bunch of coefficients (related to my previous question). The process is based on RLE where a bunch of coefficients are encoded and the runtime encoding is only focused on zeros. To cut it short this was the original array:
[200, -145, 0, 0, 0, 0, 51, 0, 0, 0, 0, 0, 0, 0, 0, -34, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 29, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Encoded into binary data that looks like this:
['000011001000', '11001>101101111<', '000010001110110011', '00010000111>1011110<', '00011000110011101', '000100011']
To avoid binary numbers that look like -10010001 (-145), I manually performed twos complement(since I couldn't find a built in way) on negative numbers. The results for the numbers in this case (-145, -34) were (101101111, 1011110).
To avoid confusion, I marked them in the array above for the purpouse of this question.
This was padded to be divisible by 8(the last element had 0's inserted into the beginning), divided into bytes and written into a file.
When I read the file, I decoded most things successfully and the number of coefficients is identical to the starting one. The problem arose with the negative values:
[200, 367, 0, 0, 0, 0, 51, 0, 0, 0, 0, 0, 0, 0, 0, 94, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 29, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Instead of -145 i got 367 and instead of -34 i got 94.
Is there any built-in way (or any kind of way) to convert bitstrings into signed values? I feel like this would fix my issue. I haven't been able to find a way and I'm stuck now.
For unsigned numbers the word size is not important because leading zeroes have no meaning there. For instance 5=101=0101=00101=0...0101. However, for two's complement the word size makes a difference because the first bit indicates negative numbers. For instance, -3=101 != 0101=5. If you don't know what the first bit is, you cannot tell whether the number is negative or not.
It seems like your encoding uses a variable word width. Since you already can decode the numbers, you already know how wide each word is.
# these variables should be set by your decoder
# in this case we read -145 encoded as 101101111
width = 9
word = 367
# add this to your decoder to fix the sign
firstBit = word >> (width - 1)
if (firstBit == 1):
leadingOnes = (-1 << width)
word = leadingOnes | word
The same could be done without the branch and in a single statement, but I think this could be slower on average for CPython and certainly less readable.
word |= -(word >> (width - 1)) << width
Of course you have to make sure that non-negative numbers are encoded with a leading 0 so that you can tell them apart from the negative ones.
According to https://pillow.readthedocs.io/en/3.1.x/handbook/concepts.html#concept-modes,
What are the difference between them?
Can we convert from one to another?
What are the examples of image for both mode?
Normally, images are RGB, which means they have 3 channels, one for red, one for green and one for blue. That normally means that each pixel takes 3 bytes of storage, one for red, one for green and one for blue.
If you have a P mode image, that means it is palettised. That means there is a palette with up to 256 different colours in it, and instead of storing 3 bytes for R, G and B for each pixel, you store 1 byte which is the index into the palette. This confers both advantages and disadvantages. The advantage is that your image requires 1/3 of the space in memory and on disk. The disadvantage is that it can only represent 256 unique colours - so you may get banding or artefacts.
If you have an L mode image, that means it is a single channel image - normally interpreted as greyscale. The L means that is just stores the Luminance. It is very compact, but only stores a greyscale, not colour.
You can tell which mode your image has by looking at:
image.mode
If your image is palettised it will be P, or PA if palettised with an alpha channel as well. If your image is greyscale it will be L, or LA if greyscale with alpha channel.
You convert between them using the convert(mode) function, e.g. to go to RGB mode, use:
image.convert('RGB')
I used the word "normally" quite a lot! Why? Because you can do abnormal things!
You can store a grey-looking image in an RGB format. All you do, is make the red component equal to the green component equal to the blue component (R=G=B) and it will appear grey but be stored in an inefficient RGB format that takes 3x the space it might otherwise need to.
You can store a grey-looking image in a P format, you just make sure all the palette entries have the R=G=B.
Here's the kicker... if you want and expect an RGB image, you should just convert to RGB on opening:
im = Image.open("image.jpg").convert('RGB')
that way you will never have problems with GIF files (which are always palettised) nor with PNG files which can be palettised and can be greyscale or RGB. You will not normally get problems with JPEG images because they are pretty much always RGB anyway.
Here's an example to demonstrate. Start with this red-blue gradient image:
Let's use IPython to look at in RGB space. First, look at the Red channel:
In [21]: im = Image.open('a.png').convert('RGB')
In [22]: np.array(im.getchannel(0))
Out[22]:
array([[255, 255, 255, ..., 255, 255, 255],
[255, 255, 255, ..., 255, 255, 255],
[254, 254, 254, ..., 254, 254, 254],
...,
[ 1, 1, 1, ..., 1, 1, 1],
[ 0, 0, 0, ..., 0, 0, 0],
[ 0, 0, 0, ..., 0, 0, 0]], dtype=uint8)
Notice it has 255 at the top because it is red, and 0 at the bottom because there is no red there.
Now let's look at the Green channel, it is 0 everywhere because there is no green.
In [23]: np.array(im.getchannel(1))
Out[23]:
array([[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]], dtype=uint8)
And finally, let's look at the Blue channel. It is 0 at the top where the image is pure Red and 255 at the bottom where the image is pure Blue.
In [24]: np.array(im.getchannel(2))
Out[24]:
array([[ 0, 0, 0, ..., 0, 0, 0],
[ 0, 0, 0, ..., 0, 0, 0],
[ 1, 1, 1, ..., 1, 1, 1],
...,
[254, 254, 254, ..., 254, 254, 254],
[255, 255, 255, ..., 255, 255, 255],
[255, 255, 255, ..., 255, 255, 255]], dtype=uint8)
Now let's look at the same image in palette mode.
# Convert to palette mode
im = Image.open('a.png').convert('P')
# Extract the palette and reshape as 256 entries of 3 RGB bytes each
In [27]: np.array(im.getpalette()).reshape(256,3)
Out[27]:
array([[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 51, 0, 0],
[102, 0, 0],
[153, 0, 0],
[204, 0, 0],
[255, 0, 0], <--- entry 15 = rgb(255,0,0) = Red
[ 0, 51, 0],
[ 51, 51, 0],
[102, 51, 0],
[153, 51, 0],
[204, 51, 0],
[255, 51, 0],
[ 0, 102, 0],
[ 51, 102, 0],
[102, 102, 0],
[153, 102, 0],
[204, 102, 0],
[255, 102, 0],
[ 0, 153, 0],
[ 51, 153, 0],
[102, 153, 0],
[153, 153, 0],
[204, 153, 0],
[255, 153, 0],
[ 0, 204, 0],
[ 51, 204, 0],
[102, 204, 0],
[153, 204, 0],
[204, 204, 0],
[255, 204, 0],
[ 0, 255, 0],
[ 51, 255, 0],
[102, 255, 0],
[153, 255, 0],
[204, 255, 0],
[255, 255, 0],
...
... up to 256 entries
Now get the indices into the palette:
In [28]: np.array(im.getchannel(0))
Out[28]:
array([[ 15, 15, 15, ..., 15, 15, 15],
[ 15, 15, 15, ..., 15, 15, 15],
[ 15, 15, 15, ..., 15, 15, 15],
...,
[190, 190, 190, ..., 190, 190, 190],
[190, 190, 190, ..., 190, 190, 190],
[190, 190, 190, ..., 190, 190, 190]], dtype=uint8)
Now you can see that the top row of the image has palette index 15, which, if you look it up in the preceding palette, you will see is Red.
Now let's look at the same image in L mode - remember L means "Luminance" which is just a fancy way of saying "brightness" on a scale of black to white, i.e. greyscale :
# Open into greyscale, or L mode
In [1]: im = Image.open('a.png').convert('L')
# Dump the pixels
In [2]: np.array(im.getchannel(0))
Out[2]:
array([[76, 76, 76, ..., 76, 76, 76],
[76, 76, 76, ..., 76, 76, 76],
[76, 76, 76, ..., 76, 76, 76],
...,
[29, 29, 29, ..., 29, 29, 29],
[29, 29, 29, ..., 29, 29, 29],
[29, 29, 29, ..., 29, 29, 29]], dtype=uint8)
So, now the top row of the image is 76 and the bottom row is 29. What are those? Well, the formula for converting RGB to L is:
L = R * 299/1000 + G * 587/1000 + B * 114/1000
So, in the top row, R=255, G=0, B=0, so the Luminance has become:
L = 255 * 299/1000 + 0 + 0
L = 76
And on the bottom row, R=0, G=0, B=255, so the Luminance has become:
L = 0 + 0 + 255 * 114/1000
L = 29
Keywords: Python, PIL, Pillow, palette, image processing, prime.
"L" mode maps to black and white pixels (and in between). "P" mode maps with a color palette. You can convert image to one of these modes.
from PIL import Image
im = Image.open("im.jpg")
im_l = im.convert('L')
im_p = im.convert('P')
im.show()
im_l.show()
im_p.show()
The "L" mode represents grayscale here.... So it can hold any of 256 shades of Gray (includes black and white as Gray shades).
The "P" mode can hold 256 different colors like red,blue, green etc....
Convertion from one another, if you mean converting images from grayscale to color or vice versa.... Yes it is possible....
Examples: 8 bit black and white image ( technically Gray scale image)are "L" and any 8 bit color images are "P" mode..
Given the following tensor (which is the result of a network [note the grad_fn]):
tensor([121., 241., 125., 1., 108., 238., 125., 121., 13., 117., 121., 229.,
161., 13., 0., 202., 161., 121., 121., 0., 121., 121., 242., 125.],
grad_fn=<MvBackward>)
Which we will define as:
xx = torch.tensor([121., 241., 125., 1., 108., 238., 125., 121., 13., 117., 121., 229.,
161., 13., 0., 202., 161., 121., 121., 0., 121., 121., 242., 125.]).requires_grad_(True)
I would like to define an operation which counts the number of occurrences of each value in such a way that the operation will output the following tensor:
tensor([2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0,
0, 7, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,
0, 1, 1])
i.e. there are 2 zeros, 1 one, 2 thirteens, etc... the total number of possible values is set upstream, but in this example is 243
So far I have tried the following approaches, which successfully produce the desired tensor, but do not do so in a way that allows computing gradients back through the network:
Attempt 1
tt = []
for i in range(243):
tt.append((xx == i).unsqueeze(0))
torch.cat(tt,dim=0).sum(dim=1)
Attempt 2
tvtensor = torch.tensor([i for i in range(243)]).unsqueeze(1).repeat(1,xx.shape[0]).float().requires_grad_(True)
(xx==tvtensor).sum(dim=1)
EDIT: Added Attempt
Attempt 3
-- Didn't really expect this to back prop, but figured I would give it a try anyway
ll = torch.zeros((1,243))
for x in xx:
ll[0,x.long()] += 1
Any help is appreciated
EDIT: As requested the end goal of this is the following:
I am using a technique for calculating structural similarity between two time sequences. One is real and the other is generated. The technique is outlined in this paper (https://link.springer.com/chapter/10.1007/978-3-642-02279-1_33) where a time series is converted to a sequence of code words and the distribution of code words (similar to the way that Bag of Words is used in NLP) is used to represent the time series. Two series are considered similar when the two signal distributions are similar. This is what the counting statistics tensor is for.
What is desired is to be able to construct a loss function which consumes this tensor and measures the distance between the real and generated signal (euclidian norm on the time domain data directly does not work well and this approach claimed better results), so that it can update the generator appropriately.
I would do it with unique method (only to count occurrences):
if you want to count the occurrences, you have to add the parameter return_counts=True
I did it in the version 1.3.1
This is the fast way to count occurrences, however is a non-differentiable operation, therefore, this method is not recommendable (anyway I have described the way to count ocurrences). To perform what you want, I think you should turn the input into a distribution by means of a differentiable function (softmax is the most used) and then, use some way to measure the distance between distributions (output and target) like cross-entropy, KL (kullback-liebler), JS or wasserstein.
You will not be able to do that as unique is simply non-differentiable operation.
Furthermore, only floating point tensors can have gradient as it's defined only for real numbers domain, not for integers.
Still, there might be another, differentiable way to do what you want to achieve, but that's a different question.
The "uniquify" operation is non-differentiable, but there might be ways to remedy this, for instance, by writing a custom operator, or by a clever combination of differentiable operators.
However, you need to ask yourself this question: what do you expect the gradient of such operation to be? Or, on a higher level, what are you trying to achieve with this operation?
I have a very large 1d array with most elements being zero while nonzero elements are all clustered around some few islands separated by many zeros: (here is a smaller version of that for the purpose of a MWE)
In [1]: import numpy as np
In [2]: A=np.array([0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,3,6,20,14,10,5,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,4,5,5,18,18,16,14,10,2,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,2,3,3,6,16,4,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0])
I want to find the median and its position (even approximately) in terms of the index corresponding to the median value of each island. Not surprisingly, I am getting zero which is not what I desire:
In [3]: np.median(A)
Out[3]: 0.0
In [4]: np.argsort(A)[len(A)//2]
Out[4]: 12
In the case of a single island of nonzero elements, to work around this caveat and meet my requirement that only nonzero elements are physically meaningful, I remove all zeros first and then take the median of the remaining elements:
In [5]: masks = np.where(A>0)
In [6]: A[masks]
Out[6]: array([ 1, 3, 6, 20, 14, 10, 5, 1])
This time, I get the median of the new array correctly, however the position (index) would not be correct as it is evident and also pointed out in the comments as being ill-defined mathematically.
In [7]: np.median(A[masks])
Out[7]: 5.5
In [8]: np.argsort(A[masks])[len(A[masks])//2]
Out[8]: 2
According to this approximation, I know that real median is located in the third index of the modified array but I would like to translate it back into the format of the original array where the position (index) of the median should be somewhere in the middle of the first island of the nonzero elements corresponding to a larger index (where indices of zeros are all counted correctly). Also answered in the comments are two suggestions made to come up with the position of the median given one island of nonzero elements in the middle of a sea of zeros. But what if there is more than one such island? How could possibly one calculate the index corresponding to median of each island in the context of the original histogram array where zeros are all counted?
I am wondering if there is any easy way to calculate the position of the median in such arrays of many zeros. If not, what else should I add to my lines of code to make that possible after knowing the position in the modified array? Your help is great appreciated.
Based on the comment "A is actually a discrete histogram with many bins", I think what you want is the median of the values being counted. If A is an integer array of counts, then an exact (but probably very inefficient, if you have values as high as 1e7) formula for the median is
np.median(np.repeat(np.arange(len(A)), A)) # Do not use if A contains very large values!
Alternatively, you can use
np.searchsorted(A.cumsum(), 0.5*A.sum())
which will be the integer part of the median.
For example:
In [157]: A
Out[157]:
array([ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 3,
6, 20, 14, 10, 5, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0])
In [158]: np.median(np.repeat(np.arange(len(A)), A))
Out[158]: 35.5
In [159]: np.searchsorted(A.cumsum(), 0.5*A.sum())
Out[159]: 35
Another example:
In [167]: B
Out[167]:
array([ 0, 0, 0, 1, 100, 21, 8, 3, 2, 1, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])
In [168]: np.median(np.repeat(np.arange(len(B)), B))
Out[168]: 4.0
In [169]: np.searchsorted(B.cumsum(), 0.5*B.sum())
Out[169]: 4