I'm writing a script to chroma key (green screen) and composite some videos using Python and PIL (pillow). I can key the 720p images, but there's some left over green spill. Understandable but I'm writing a routine to remove that spill...however I'm struggling with how long it's taking. I can probably get better speeds using numpy tricks, but I'm not that familiar with it. Any ideas?
Here's my despill routine. It takes a PIL image and a sensitivity number but I've been leaving that at 1 so far...it's been working well. I'm coming in at just over 4 seconds for a 720p frame to remove this spill. For comparison, the chroma key routine runs in about 2 seconds per frame.
def despill(img, sensitivity=1):
"""
Blue limits green.
"""
start = time.time()
print '\t[*] Starting despill'
width, height = img.size
num_channels = len(img.getbands())
out = Image.new("RGBA", img.size, color=0)
for j in range(height):
for i in range(width):
#r,g,b,a = data[j,i]
r,g,b,a = img.getpixel((i,j))
if g > (b*sensitivity):
out_g = (b*sensitivity)
else:
out_g = g
# end if
out.putpixel((i,j), (r,out_g,b,a))
# end for
# end for
out.show()
print '\t[+] done.'
print '\t[!] Took: %0.1f seconds' % (time.time()-start)
exit()
return out
# end despill
Instead of putpixel, I tried to write the output pixel values to a numpy array then convert the array to a PIL image, but that was averaging just over 5 seconds...so this was faster somehow. I know putpixel isn't the snappiest option but I'm at a loss...
putpixel is slow, and loops like that are even slower, since they are run by the Python interpreter, which is slow as hell. The usual solution is to convert immediately the image to a numpy array and solve the problem with vectorized operations on it, which run in heavily optimized C code. In your case I would do something like:
arr = np.array(img)
g = arr[:,:,1]
bs = arr[:,:,2]*sensitivity
cond = g>bs
arr[:,:,1] = cond*bs + (~cond)*g
out = Image.fromarray(arr)
(it may not be correct and I'm sure it can be optimized way better, this is just a sketch)
Related
I have a nested loop that has to loop through a huge amount of data.
Assuming a data frame with random values with a size of 1000,000 rows each has an X,Y location in 2D space. There is a window of 10 length that go through all the 1M data rows one by one till all the calculations are done.
Explaining what the code is supposed to do:
Each row represents a coordinates in X-Y plane.
r_test is containing the diameters of different circles of investigations in our 2D plane (X-Y plane).
For each 10 points/rows, for every single diameter in r_test, we compare the distance between every point with the remaining 9 points and if the value is less than R we add 2 to H. Then we calculate H/(N**5) and store it in c_10 with the index corresponding to that of the diameter of investigation.
For this first 10 points finally when the loop went through all those diameters in r_test, we read the slope of the fitted line and save it to S_wind[ii]. So the first 9 data points will have no value calculated for them thus giving them np.inf to be distinguished later.
Then the window moves one point down the rows and repeat this process till S_wind is completed.
What's a potentially better algorithm to solve this than the one I'm using? in python 3.x?
Many thanks in advance!
import numpy as np
import pandas as pd
####generating input data frame
df = pd.DataFrame(data = np.random.randint(2000, 6000, (1000000, 2)))
df.columns= ['X','Y']
####====creating upper and lower bound for the diameter of the investigation circles
x_range =max(df['X']) - min(df['X'])
y_range = max(df['Y']) - min(df['Y'])
R = max(x_range,y_range)/20
d = 2
N = 10 #### Number of points in each window
#r1 = 2*R*(1/N)**(1/d)
#r2 = (R)/(1+d)
#r_test = np.arange(r1, r2, 0.05)
##===avoiding generation of empty r_test
r1 = 80
r2= 800
r_test = np.arange(r1, r2, 5)
S_wind = np.zeros(len(df['X'])) + np.inf
for ii in range (10,len(df['X'])): #### maybe the code run slower because of using len() function instead of a number
c_10 = np.zeros(len(r_test)) +np.inf
H = 0
C = 0
N = 10 ##### maybe I should also remove this
for ind in range(len(r_test)):
for i in range (ii-10,ii):
for j in range(ii-10,ii):
dd = r_test[ind] - np.sqrt((df['X'][i] - df['X'][j])**2+ (df['Y'][i] - df['Y'][j])**2)
if dd > 0:
H += 1
c_10[ind] = (H/(N**2))
S_wind[ii] = np.polyfit(np.log10(r_test), np.log10(c_10), 1)[0]
You can use numpy broadcasting to eliminate all of the inner loops. I'm not sure if there's an easy way to get rid of the outermost loop, but the others are not too hard to avoid.
The inner loops are comparing ten 2D points against each other in pairs. That's just dying for using a 10x10x2 numpy array:
# replacing the `for ind` loop and its contents:
points = np.hstack((np.asarray(df['X'])[ii-10:ii, None], np.asarray(df['Y'])[ii-10:ii, None]))
differences = np.subtract(points[None, :, :], points[:, None, :]) # broadcast to 10x10x2
squared_distances = (differences * differences).sum(axis=2)
within_range = squared_distances[None,:,:] < (r_test*r_test)[:, None, None] # compare squares
c_10 = within_range.sum(axis=(1,2)).cumsum() * 2 / (N**2)
S_wind[ii] = np.polyfit(np.log10(r_test), np.log10(c_10), 1)[0] # this is unchanged...
I'm not very pandas savvy, so there's probably a better way to get the X and Y values into a single 2-dimensional numpy array. You generated the random data in the format that I'd find most useful, then converted into something less immediately useful for numeric operations!
Note that this code matches the output of your loop code. I'm not sure that's actually doing what you want it to do, as there are several slightly strange things in your current code. For example, you may not want the cumsum in my code, which corresponds to only re-initializing H to zero in the outermost loop. If you don't want the matches for smaller values of r_test to be counted again for the larger values, you can skip that sum (or equivalently, move the H = 0 line to in between the for ind and the for i loops in your original code).
I an running OCR on bank cheques using pyimagesearch tutorial to detect micr code. The code used in the tutorial detects group contours & character contours from a reference image containing symbols.
In the tutorial when finding the contours for symbol below
the code uses an built-in python iterator to iterate over the contours (here 3 seperate contours) and combined to give a character for recognition purposes.
But in the cheque dataset that I use, I have the symbol with low resolution
The actual bottom of the cheque is :
which causes the iterator to consider the contour-2 & contour-3 as a single contour. Due to this the iterator iterates over the character following the above symbol (here '0') and prepares a incorrect template to match with the reference symbols. You can see the code below for better understanding.
I know here noise in the image is a factor, but is it possible to reduce the noise & also find the exact contour to detect the symbol?
I tried using noise reduction techniques like cv2.fastNlMeansDenoising & cv2.GaussianBlur before cv2.findContours step the contours 2&3 are detected as single contour instead of 2 seperate contours.
Also I tried altering the `cv2.findContours' parameters
Below is the working code where the characters are iterated for better understanding of python builtin iterator:
def extract_digits_and_symbols(image, charCnts, minW=5, minH=10):
# grab the internal Python iterator for the list of character
# contours, then initialize the character ROI and location
# lists, respectively
charIter = charCnts.__iter__()
rois = []
locs = []
# keep looping over the character contours until we reach the end
# of the list
while True:
try:
# grab the next character contour from the list, compute
# its bounding box, and initialize the ROI
c = next(charIter)
(cX, cY, cW, cH) = cv2.boundingRect(c)
roi = None
# check to see if the width and height are sufficiently
# large, indicating that we have found a digit
if cW >= minW and cH >= minH:
# extract the ROI
roi = image[cY:cY + cH, cX:cX + cW]
rois.append(roi)
cv2.imshow('roi',roi)
cv2.waitKey(0)
locs.append((cX, cY, cX + cW, cY + cH))
# otherwise, we are examining one of the special symbols
else:
# MICR symbols include three separate parts, so we
# need to grab the next two parts from our iterator,
# followed by initializing the bounding box
# coordinates for the symbol
parts = [c, next(charIter), next(charIter)]
(sXA, sYA, sXB, sYB) = (np.inf, np.inf, -np.inf,
-np.inf)
# loop over the parts
for p in parts:
# compute the bounding box for the part, then
# update our bookkeeping variables
# c = next(charIter)
# (cX, cY, cW, cH) = cv2.boundingRect(c)
# roi = image[cY:cY+cH, cX:cX+cW]
# cv2.imshow('symbol', roi)
# cv2.waitKey(0)
# roi = None
(pX, pY, pW, pH) = cv2.boundingRect(p)
sXA = min(sXA, pX)
sYA = min(sYA, pY)
sXB = max(sXB, pX + pW)
sYB = max(sYB, pY + pH)
# extract the ROI
roi = image[sYA:sYB, sXA:sXB]
cv2.imshow('symbol', roi)
cv2.waitKey(0)
rois.append(roi)
locs.append((sXA, sYA, sXB, sYB))
# we have reached the end of the iterator; gracefully break
# from the loop
except StopIteration:
break
# return a tuple of the ROIs and locations
return (rois, locs)
edit: contour 2 & 3 instead of contours 1 & 2
Try to find the right threshold value, instead of using cv2.THRESH_OTSU. It seems should be possible to find a suitable threshold from the provided example. If you can't find the threshold value that works for all images, you can try morphological closing on the threshold result with structuring element with 1-pixel width.
Edit (steps):
For threshold, you need to find appropriate value by hand, in your image threhsold value 100 seems to work:
i = cv.imread('image.png')
g = cv.cvtColor(i, cv.COLOR_BGR2GRAY)
_, tt = cv.threshold(g, 100, 255, cv.THRESH_BINARY_INV)
as for closing variant:
_, t = cv.threshold(g, 0,255,cv.THRESH_BINARY_INV | cv.THRESH_OTSU)
kernel = np.ones((12,1), np.uint8)
c = cv.morphologyEx(t, cv.MORPH_OPEN, kernel)
Note that I used import cv2 as cv. I also used opening instead of closing since in the example they inverted colors during thresholding
I'm pretty new to python and Opencv, but I have a few pieces from cv2 and random in mind for a simple test program to make sure I understood how these libraries worked.
I'm trying to create a program that effectively generates colored "snow", similar to what an old fashioned television shows when it has no signal.
Basically I generate a random color with random.randint(-1,256) to get a value between 0 and 255. I do it three times and store each in a different variable, randB/G/R. Then I do it twice more for coordinates randX/Y, using img.shape to get variables for width and height for the max number.
I don't think my variables are being interpreted as strings. If I quickly break the loop and print my variables, no errors are shown. If I remove the randX and randY variables and specify fixed coordinates or a range of [X1:Y1, X2:Y2] it doesn't crash.
import cv2
import numpy as np
import random
img = cv2.imread('jake_twitch.png', cv2.IMREAD_COLOR)
height, width, channels = img.shape
while True:
randB = (random.randint(-1,256))
randG = (random.randint(-1,256))
randR = (random.randint(0,256))
randX = (random.randint(0,width))
randY = (random.randint(0,height))
img[randX,randY] = [randB,randG,randR]
cv2.imshow('Snow', img)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cv2.imwrite('Snow.png', img)
cv2.destroyAllWindows
I would expect my code to run indefinitely coloring pixels random colors within a specified "box" defined by the width and height variables from img.shape.
it seems to start doing that, but If the program runs for more than about a second it crashes and spits out this error
"IndexError: index 702 is out of bounds for axis 1 with size 702"
Your image is width and height pixels wide - but the corresponding indexes run from 0..width-1 and 0..height-1
The randint function returns inclusive limits - so
random.randint(0,width)
might give you width ... which is 1 too big:
random.randint(a, b)
Return a random integer N such that a <= N <= b. Alias for randrange(a, b+1).
Use
randX = (random.randint(0,width-1))
randY = (random.randint(0,height-1))
instead.
Or change it to use random.randrange(0, width) or random.choice(range(width)) - both omit the upper limit value.
I'm new to OpenCV and I'm trying to get the pixels of a circle from an image.
For example, I draw a circle on a random image:
import cv2
raw_img = cv2.imread('sample_picture.png')
x = 50
y = 50
rad = 20
cv2.circle(raw_img,(x,y),rad,(0,255,0),-1)
cv2.imshow('output', raw_img)
cv2.waitKey(0)
cv2.destroyAllWindows()
The output shows an image with a circle.
However, I want to be able to get all the pixel on the circle back in the form of an array. Is there any way to do this? I know I can get the approximate coordinates from the circle formula, but it will involve a lot of decimal calculations, and I'm pretty sure that the function cv2.circle() has already calculated the pixel, so is there a way to get it out from the function itself instead of calculating my self?
Also, if it is possible I would like to get the pixel of an ellipse using cv2.ellipse() back as an array of coordinates. But this time, I want to get the pixel only from a part of an ellipse (from a certain angle to another angle, which I can specify in the parameter of cv2.ellipse()).
Thank you.
You can achieve what you are looking for by using the numpy function:
numpy.where(condition[, x, y])
Detailed explanation of function in link :https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.where.html
In your case, you would want to it to return the coordinates that has non-zero values. Using this method, you can draw anything on an empty array and it will return all rows and columns corresponding to non-zeros.
It will return the index of the array that satisfies the condition you set. Below is a code showing an example of the usage.
import cv2
import numpy as np
raw_img = cv2.imread('sample_picture.png')
x = 50
y = 50
rad = 20
cv2.circle(raw_img,(x,y),rad,(0,255,0),-1)
# Here is where you can obtain the coordinate you are looking for
combined = raw_img[:,:,0] + raw_img[:,:,1] + raw_img[:,:,2]
rows, cols, channel = np.where(combined > 0)
cv2.imshow('output', raw_img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Please help my poor knowledge of signal processing.
I want to smoothen some data. Here is my code:
import numpy as np
from scipy.signal import butter, filtfilt
def testButterworth(nyf, x, y):
b, a = butter(4, 1.5/nyf)
fl = filtfilt(b, a, y)
return fl
if __name__ == '__main__':
positions_recorded = np.loadtxt('original_positions.txt', delimiter='\n')
number_of_points = len(positions_recorded)
end = 10
dt = end/float(number_of_points)
nyf = 0.5/dt
x = np.linspace(0, end, number_of_points)
y = positions_recorded
fl = testButterworth(nyf, x, y)
I am pretty satisfied with results except one point:
it is absolutely crucial to me that the start and end point in returned values equal to the start and end point of input. How can I introduce this restriction?
UPD 15-Dec-14 12:04:
my original data looks like this
Applying the filter and zooming into last part of the graph gives following result:
So, at the moment I just care about the last point that must be equal to original point. I try to append copy of data to the end of original list this way:
the result is as expected even worse.
Then I try to append data this way:
And the slice where one period ends and next one begins, looks like that:
To do this, you're always going to cheat somehow, since the true filter applied to the true data doesn't behave the way you require.
One of the best ways to cheat with your data is to assume it's periodic. This has the advantages that: 1) it's consistent with the data you actually have and all your changing is to append data to the region you don't know about (so assuming it's periodic as as reasonable as anything else -- although may violate some unstated or implicit assumptions); 2) the result will be consistent with your filter.
You can usually get by with this by appending copies of your data to the beginning and end of your real data, or just small pieces, depending on your filter.
Since the FFT assumes that the data is periodic anyway, that's often a quick and easy approach, and is fully accurate (whereas concatenating the data is an estimation of an infinitely periodic waveform). Here's an example of the FFT approach for a step filter.
import numpy as np
import matplotlib.pyplot as plt
x = np.arange(0, 128)
y = (np.sin(.22*(x+10))>0).astype(np.float)
# filter
y2 = np.fft.fft(y)
f0 = np.fft.fftfreq(len(x))
y2[(f0<-.25) | (f0>.25)] = 0
y3 = abs(np.fft.ifft(y2))
plt.plot(x, y)
plt.plot(x, y3)
plt.xlim(-10, 140)
plt.ylim(-.1, 1.1)
plt.show()
Note how the end points bend towards each other at either end, even though this is not consistent with the periodicity of the waveform (since the segments at either end are very truncated). This can also be seen by adjusting waveform so that the ends are the same (here I used x+30 instead of x+10, and here the ends don't need to bend to match-up so they stay at level with the end of the data.
Note, also, to have the endpoints actually be exactly equal you would have to extend this plot by one point (at either end), since it periodic with exactly the wavelength of the original waveform. Doing this is not ad hoc though, and the result will be entirely consistent with your analysis, but just representing one extra point of what was assumed to be infinite repeats all along.
Finally, this FFT trick works best with waveforms of length 2n. Other lengths may be zero padded in the FFT. In this case, just doing concatenations to either end as I mentioned at first might be the best way to go.
The question is how to filter data and require that the left endpoint of the filtered result matches the left endpoint of the data, and same for the right endpoint. (That is, in general, the filtered result should be close to most of the data points, but not necessarily exactly match any of them, but what if you need a match at both endpoints?)
To make the filtered result exactly match the endpoints of a curve, one could add a padding of points at either end of the curve and adjust the y-position of this padding so that the endpoints of the valid part of the filter exactly matched the end points of the original data (without the padding).
In general, this can be done by either iterating towards a solution, adjusting the padding y-position until the ends line up, or by calculating a few values and then interpolating to determine the y-positions that would be required for the matched endpoints. I'll do the second approach.
Here's the code I used, where I simulated the data as a sine wave with two flat pieces on either side (note, that these flat pieces are not the padding, but I'm just trying to make data that looks a bit like the OPs).
import numpy as np
from scipy.signal import butter, filtfilt
import matplotlib.pyplot as plt
#### op's code
def testButterworth(nyf, x, y):
#b, a = butter(4, 1.5/nyf)
b, a = butter(4, 1.5/nyf)
fl = filtfilt(b, a, y)
return fl
def do_fit(data):
positions_recorded = data
#positions_recorded = np.loadtxt('original_positions.txt', delimiter='\n')
number_of_points = len(positions_recorded)
end = 10
dt = end/float(number_of_points)
nyf = 0.5/dt
x = np.linspace(0, end, number_of_points)
y = positions_recorded
fx = testButterworth(nyf, x, y)
return fx
### simulate some data (op should have done this too!)
def sim_data():
t = np.linspace(.1*np.pi, (2.-.1)*np.pi, 100)
y = np.sin(t)
c = np.ones(10, dtype=np.float)
z = np.concatenate((c*y[0], y, c*y[-1]))
return z
### code to find the required offset padding
def fit_with_pads(v, data, n=1):
c = np.ones(n, dtype=np.float)
z = np.concatenate((c*v[0], data, c*v[1]))
fx = do_fit(z)
return fx
def get_errors(data, fx):
n = (len(fx)-len(data))//2
return np.array((fx[n]-data[0], fx[-n]-data[-1]))
def vary_padding(data, span=.005, n=100):
errors = np.zeros((4, n)) # Lpad, Rpad, Lerror, Rerror
offsets = np.linspace(-span, span, n)
for i in range(n):
vL, vR = data[0]+offsets[i], data[-1]+offsets[i]
fx = fit_with_pads((vL, vR), data, n=1)
errs = get_errors(data, fx)
errors[:,i] = np.array((vL, vR, errs[0], errs[1]))
return errors
if __name__ == '__main__':
data = sim_data()
fx = do_fit(data)
errors = vary_padding(data)
plt.plot(errors[0], errors[2], 'x-')
plt.plot(errors[1], errors[3], 'o-')
oR = -0.30958
oL = 0.30887
fp = fit_with_pads((oL, oR), data, n=1)[1:-1]
plt.figure()
plt.plot(data, 'b')
plt.plot(fx, 'g')
plt.plot(fp, 'r')
plt.show()
Here, for the padding I only used a single point on either side (n=1). Then I calculate the error for a range of values shifting the padding up and down from the first and last data points.
For the plots:
First I plot the offset vs error (between the fit and the desired data value). To find the offset to use, I just zoomed in on the two lines to find the x-value of the y zero crossing, but to do this more accurately, one could calculate the zero crossing from this data:
Here's the plot of the original "data", the fit (green) and the adjusted fit (red):
and zoomed in the RHS:
The important point here is that the red (adjusted fit) and blue (original data) endpoints match, even though the pure fit doesn't.
Is this a valid approach? Of the various options, this seems the most reasonable since one isn't usually making any claims about the data that isn't being shown, and also for show region has an accurately applied filter. For example, FFTs usually assume the data is zero or periodic beyond the boundaries. Certainly, though, to be precise one should explain what was done.