I am trying to build a tool that would recognize poker cards from an online site.
I thought the task would be trivial.
Harvest all possible cards, paste them in one image, get an "unknown" card, run template matching on all_cards_image, harvest the points where it is matched, compare it to a dictionary with (x,y) = "Ah" and voila.
But template matching gives such hit and miss results, some single cards are not recognized at all, some single cards are recognized as 4 of hearts, 5 of hearts and 6 of hearts simultaneously. 8 of clubs is recognized as 8 of spades and so on. When no card is present then it matches the whole template image.
I could not find threshold that would lead to satisfactory results.
I am attaching the whole code as well as the images.
It is clear to me that I could go the route of recognizing only rank of the card, maybe on black and white image and masking the suits for color (blue, green, red, black) and then matching them afterwards. I was just hoping there is a simple template matching solution without jumping through hoops.
First image with all the cards (all_cards.jpg) in code:
Second image (random screenshot) with all other details changed to white so as not to make the image too big, named "Screenshot_to_test_card_recognition_reduced.png" in code:
import cv2
import numpy as np
import os
# defining constants for positions, card width and height in a screenshot
CARD_HEIGHT = 59
CARD_WIDTH = 90
HORIZONTAL_SPACE = 5
HORIZONTAL_OFFSET_2 = 1463
VERTICAL_OFFSET_2 = 1047
FIRST_CARD_X = 955
FIRST_CARD_Y = 370
FLOPS_STARTING_POINTS = [(FIRST_CARD_X, FIRST_CARD_Y), (FIRST_CARD_X + HORIZONTAL_OFFSET_2, FIRST_CARD_Y), \
(FIRST_CARD_X, FIRST_CARD_Y + VERTICAL_OFFSET_2), \
(FIRST_CARD_X + HORIZONTAL_OFFSET_2, FIRST_CARD_Y + VERTICAL_OFFSET_2)]
# show image function
def show_image(img, title = "Unnamed"):
cv2.imshow(title, img)
cv2.waitKey(0)
cv2.destroyAllWindows()
# INSERT FOLDER WITH FILES
saved_folder = "D:\\WinPython 32 3.4\\WinPython-32bit-3.4.4.6Qt5\\notebooks\\Named cards"
template_image = os.path.join(saved_folder, 'all_cards.jpg')
screenshot = os.path.join(saved_folder, 'Screenshot_to_test_card_recognition_reduced.png')
img_rgb = cv2.imread(screenshot)
img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
CORRECTION_FOR_SEARCH_WIDTH = 0 #60
CORRECTION_FOR_SEARCH_HEIGTH = 0 #20
# LOOP THROUGH ALL FIVE CARDS FOR EACH FLOP ON 4 TABLES
for point in FLOPS_STARTING_POINTS:
flop_starting_x, flop_starting_y = point
for card in range (0,5):
start_x = flop_starting_x + ((HORIZONTAL_SPACE) * card) + ((CARD_WIDTH) * card)
end_x = start_x + CARD_WIDTH - CORRECTION_FOR_SEARCH_WIDTH
start_y = flop_starting_y
end_y = start_y + CARD_HEIGHT - CORRECTION_FOR_SEARCH_HEIGTH
template = img_gray[start_y:end_y,start_x:end_x]
w, h = template.shape[::-1]
show_image(template)
all_cards_image = cv2.imread(template_image)
grey_all_cards_image = cv2.cvtColor(all_cards_image, cv2.COLOR_BGR2GRAY)
res = cv2.matchTemplate(grey_all_cards_image,template,cv2.TM_CCOEFF_NORMED)
threshold = 0.85
loc = np.where( res >= threshold)
for pt in zip(*loc[::-1]):
cv2.rectangle(all_cards_image, pt, (pt[0] + w, pt[1] + h), (0,0,255), 2)
print(loc)
show_image(all_cards_image)
Related
I have multiple image templates of some objects and my app should allow the user to input random images with random size and I want to use OpenCV to create a method that will take a template and the user image and return the percentage of the similarity or the matching in some cases
Example:
the user input is only a ball in the street
and the three templates is the ball, apple, and car images so the results should be 1 for the ball template because the image contains ball and zero with car template and between 0and 1 for the apple template because the apple has some similarity with ball
Any snippet code or hints are appreciated I can work on the code if it needs any improvements
I have this following code that will measure the matching but it not return the similarity on the matching percentage of the template in the image
import cv2
img = cv2.imread('big_image.jpg', 0)
template = cv2.imread('ball.jpg', 0)
h, w = template.shape
img2 = img.copy()
result = cv2.matchTemplate(img2, template, cv2.TM_SQDIFF)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)
location = min_loc
bottom_right = (location[0] + w + 10, location[1] + h + 10)
cv2.rectangle(img2, location, bottom_right, 255, 2)
cv2.imshow('Match', img2)
cv2.waitKey(0)
cv2.destroyAllWindows()
I will add any more info if need it
I am facing a problem in segmenting characters from a license plate image.
I have applied following method to extract license plate characters, after this I will input every character into a OCR (tesseract):
Rgb2gray.
Canny edge detection and dilation.
Mark and filter areas.
Extract a single characters image.
The results are attached below:
On the first column work well.
At the second column there is a Chinese plate, and it is difficult to correctly identify every character.
At the third column there is another plate, which it was rotated through the hough transform because it was from another angle, however, it cannot detect any character.
I'm working here, on Google Colab.
What can I improve? any advice?
These are the image im working at this link
I think the problem it maybe at Canny edge detection:
img3 = feature.canny(img2, sigma=3)
img4 = morphology.dilation(img3)
Or filtering areas:
label_img = measure.label(img4)
regions = measure.regionprops(label_img)
fig, ax = plt.subplots()
ax.imshow(img, cmap=plt.cm.gray)
def in_bboxes(bbox, bboxes):
for bb in bboxes:
minr0, minc0, maxr0, maxc0 = bb
minr1, minc1, maxr1, maxc1 = bbox
if minr1 >= minr0 and maxr1 <= maxr0 and minc1 >= minc0 and maxc1 <= maxc0:
return True
return False
bboxes = []
for props in regions:
y0, x0 = props.centroid
minr, minc, maxr, maxc = props.bbox
if maxc - minc > img4.shape[1] / 7 or maxr - minr < img4.shape[0] / 3:
continue
bbox = [minr, minc, maxr, maxc]
if in_bboxes(bbox, bboxes):
continue
if abs(y0 - img4.shape[0] / 2) > img4.shape[0] / 4:
continue
bboxes.append(bbox)
bx = (minc, maxc, maxc, minc, minc)
by = (minr, minr, maxr, maxr, minr)
ax.plot(bx, by, '-r', linewidth=2)
For apply projection before steps.
I have the following problem, I have a pair of two images one historical and one present-day satellite image and as the historical image covers a smaller area I want to crop the satellite images. Here one can see the code I wrote for this:
import numpy as np
import cv2
import os
import imutils
import math
entries = os.listdir('../')
refImage = 0
histImages = []
def loadImage(index):
referenceImage = cv2.imread("../" + 'ref_' + str(index) + '.png')
top = int(0.5 * referenceImage.shape[0]) # shape[0] = rows
bottom = top
left = int(0.5 * referenceImage.shape[1]) # shape[1] = cols
right = left
referenceImage = cv2.copyMakeBorder(referenceImage, top, bottom, left, right, cv2.BORDER_CONSTANT, None, (0,0,0))
counter = 0
for entry in entries:
if entry.startswith("image_"+str(index)):
refImage = referenceImage.copy()
histImage = cv2.imread("../" + entry)
#histImages.append(img)
points = np.loadtxt("H2OPM/"+"CP_"+ entry[6:9] + ".txt", delimiter=",")
vector_image1 = [points[0][0] - points[1][0], points[0][1] - points[1][1]] #hist
vector_image2 = [points[0][2] - points[1][2], points[0][3] - points[1][3]] #ref
angle = angle_between(vector_image1, vector_image2)
hhist, whist, chist = histImage.shape
rotatedImage = imutils.rotate(refImage, angle)
x = int(points[0][2] - points[0][0])
y = int(points[1][2] - points[1][0])
crop_img = rotatedImage[x+left:x+left+hhist, y+top:y+top+whist]
print("NewImageWidth:", (y+top+whist)-(y+top),(x+left+hhist)-(x+left))
print(entry)
print(x,y)
counter += 1
#histImage = cv2.line(histImage, (points[0][0], ), end_point, color, thickness)
cv2.imwrite("../matchedImages/"+'image_' + str(index) + "_" + str(counter) + '.png' ,histImage)
#rotatedImage = cv2.line(rotatedImage, (), (), (0, 255, 0), 9)
cv2.imwrite("../matchedImages/"+'ref_' + str(index) + "_" + str(counter) + '.png' ,crop_img)
First, I load the original satellite image and pad it so I don't lose information due to the rotation, second, I load one of the matched historical images as well as the matched keypoints of the two images (i.e. a list of x_hist, y_hist, x_present_day, y_present_day). Third, I compute the rotation angle between the two images (which works) and fourth, I crop the image (and fifth, I save the images).
Problem: As stated the rotation works fine, but my program ends up cropping the wrong part of the image.
I think that, due to the rotation, the boundaries (i.e. left, right, top, bottom) are no longer correct and I think this is where my problem lies, but I am not sure how to fix this problem.
Information that might help:
The images are both scaled the same way (so one pixel = approx. 1m)
I have at least 6 keypoints for each image
I haven't looked at your code yet but would it be due to you mixing up the x's and y's ? Check the OpenCV documentation to make sure the variables you import are in the correct order.
During my limited time and experience with opencv, it is quite weird because sometimes, it asks for for example, BGR instead of RGB values. (In my programme, not yours)
Also, you seem to have a bunch of lists, make sure the list[x][y] is not mixed up as list[y][x]
So I found the error in my computation. The bounding boxes of the cutout area were wrongly converted into the present-day image.
So this:
x = int(points[0][2] - points[0][0])
y = int(points[1][2] - points[1][0])
was swapped with this:
v = [pointBefore[0],pointBefore[1],1]
# Perform the actual rotation and return the image
calculated = np.dot(m,v)
newPoint = (int(calculated[0]- points[0][0]),int(calculated[1]- points[0][1]))
where m(=M) is from the transformation:
def rotate_bound(image, angle):
# grab the dimensions of the image and then determine the
# center
(h, w) = image.shape[:2]
(cX, cY) = (w // 2, h // 2)
# grab the rotation matrix (applying the negative of the
# angle to rotate clockwise), then grab the sine and cosine
# (i.e., the rotation components of the matrix)
M = cv2.getRotationMatrix2D((cX, cY), -angle, 1.0)
cos = np.abs(M[0, 0])
sin = np.abs(M[0, 1])
# compute the new bounding dimensions of the image
nW = int((h * sin) + (w * cos))
nH = int((h * cos) + (w * sin))
# adjust the rotation matrix to take into account translation
M[0, 2] += (nW / 2) - cX
M[1, 2] += (nH / 2) - cY
# perform the actual rotation and return the image
return cv2.warpAffine(image, M, (nW, nH)), M
Thanks.
I'm currently trying to create a detector of aphids (green and rose) on plants but only using "classic" image processing technique (no neural network).
Here are an image I'm working on:
'aphids.jpg'
I'm working on a code (see below). If you apply it on the image you should have the plants alone. My problem is that I want to isolate the aphids that can be seen on the plants. There are a lot of them but I just want to detect the biggest or the more obvious.
On the code there is an "edges_detect" function I'm currently working on. One of the problem I have is that I can detect some of the aphids as contour but it will also take simple lines...
I tried to drop those line using the hierarchy of contour but it seems those line have inner contour so I can't easily delete them.
I also tried the adjust_gamma and contrast, but it doesn't give that much result.
I'm looking for more ideas. What would you try ?
Thank you in advance !
Here is the code:
import cv2
import numpy as np
import matplotlib.pyplot as plt
def adjust_gamma(image, gamma=1.0):
# build a lookup table mapping the pixel values [0, 255] to
# their adjusted gamma values
invGamma = 1.0 / gamma
table = np.array([((i / 255.0) ** invGamma) * 255
for i in np.arange(0, 256)]).astype("uint8")
# apply gamma correction using the lookup table
return cv2.LUT(image, table)
def adjust_contrast(image,alpha=1.0,beta=0):
new = np.zeros(image.shape,image.dtype)
for y in range(image.shape[0]):
for x in range(image.shape[1]):
for c in range(image.shape[2]):
new[y,x,c] = np.clip(alpha*image[y,x,c]+beta,0,255)
return(new)
def img_process(img):
(h1, w1) = img.shape[:2]
center = (w1 / 2, h1 / 2)
blur = cv2.GaussianBlur(img.copy(),(5,5),0)
hsv = cv2.cvtColor(blur,cv2.COLOR_BGR2HSV)
#image = img.copy()
#Boundaries to separate plants from the image
l_bound = np.array([20,0,0])
h_bound = np.array([90,250,170])#green
mask = cv2.inRange(hsv,l_bound,h_bound)
res = cv2.bitwise_and(img,img,mask=mask)
#Find contour plants
cnt,_ = cv2.findContours(mask,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
sort_cnt = sorted(cnt,key=cv2.contourArea,reverse=True)
cnt = [sort_cnt[i] for i in range(len(sort_cnt)) if cv2.contourArea(sort_cnt[i])>300]
cv2.drawContours(res, cnt, -1, (0,255,0), -1)
#Inverse mask to have only the plant in the image
mask2 = cv2.inRange(res,np.array([0,0,0]),np.array([250,250,250]))
mask2 = cv2.bitwise_not(mask2)
res2 = cv2.bitwise_and(img,img,mask=mask2)
#Augment bright/contrast
res2=res2*1.45
res2=res2.astype('uint8')
#Crop
res2 = res2[:-50,int(center[0]-300):int(center[0]+550)]
return res2
def edge_detec(img):
(h1, w1) = img.shape[:2]
center = (w1 / 2, h1 / 2)
blur = cv2.GaussianBlur(img.copy(),(5,5),0)
gray = cv2.cvtColor(blur,cv2.COLOR_BGR2GRAY)
edges = cv2.Canny(gray,30,70,apertureSize = 3)
edges = edges[:-50,int(center[0]-300):int(center[0]+550)]
#kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(3,3))
#edges = cv2.morphologyEx(edges, cv2.MORPH_GRADIENT, kernel)
cnt,hierarchy = cv2.findContours(edges,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
cnt = sorted(cnt,key=cv2.contourArea,reverse=True)
listArea = list(map(cv2.contourArea,cnt))
sort_cnt = [x for x in cnt if cv2.contourArea(x)>10]
cv2.drawContours(edges, sort_cnt, -1, (0,255,0), -1)
return edges,center,img
### Debut programme
img = cv2.imread('051.jpg')
while True:
##Put processing function here
img_mod = img_process(img)
cv2.imshow('img',img_mod)
if cv2.waitKey(1) & 0xFF == 27:
break
cv2.destroyAllWindows()
Template matching in OpenCV is great. And you can pass a mask to cv2.minMaxLoc so that you only search (sort of) in part of the image for the template you want. You can also use a mask at the matchTemplate operation, but this only masks the template.
I want to find a template and I want to be assured that this template is within some other region of my image.
Calculating the mask for minMaxLoc seems kind of heavy. That is, calculating an accurate mask feels heavy. If you calculate a mask the easy way, it ignores the size of the template.
Examples are in order. My input images are show below. They're a bit contrived. I want to find the candy bar, but only if it's completely inside the white circle of the clock face.
clock1
clock2
template
In clock1, the candy bar is inside the circular clock face and it's a "PASS". But in clock2, the candy bar is only partially inside the face and I want it to be a "FAIL". Here's a code sample for doing it the easy way. I use cv.HoughCircles to find the clock face.
import numpy as np
import cv2
img = cv2.imread('clock1.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
template = cv2.imread('template.png')
t_h, t_w = template.shape[0:2] # template height and width
# find circle in gray image using Hough transform
circles = cv2.HoughCircles(gray, method = cv2.HOUGH_GRADIENT, dp = 1,
minDist = 150, param1 = 50, param2 = 70,
minRadius = 131, maxRadius = 200)
i = circles[0,0]
x0 = i[0]
y0 = i[1]
r = i[2]
# display circle on color image
cv2.circle(img,(x0, y0), r,(0,255,0),2)
# do the template match
result = cv2.matchTemplate(img, template, cv2.TM_CCOEFF_NORMED)
# finally, here is the part that gets tricky. we want to find highest
# rated match inside circle and we'd like to use minMaxLoc
# make mask by drawing circle on zero array
mask = np.zeros(result.shape, dtype = np.uint8) # minMaxLoc will throw
# error w/o np.uint8
cv2.circle(mask, (x0, y0), r, color = 1, thickness = -1)
# call minMaxLoc
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result, mask = mask)
# draw found rectangle on img
if max_val > 0.4: # use 0.4 as threshold for finding candy bar
cv2.rectangle(img, max_loc, (max_loc[0]+t_w, max_loc[1]+t_h), (0,255,0), 4)
cv2.imwrite('output.jpg', img)
output using clock1
output using clock2
finds candy bar even
though part of it is outside circle
So to properly make a mask, I use a bunch of NumPy operations. I make four separate masks (one for each corner of the template bounding box) and then AND them together. I'm not aware of any convenience functions in OpenCV that would do the mask for me. I'm a little nervous that all of the array operations will be expensive. Is there a better way to do this?
h, w = result.shape[0:2]
# make arrays that hold x,y coords
grid = np.indices((h, w))
x = grid[1]
y = grid[0]
top_left_mask = np.hypot(x - x0, y - y0) - r < 0
top_right_mask = np.hypot(x + t_w - x0, y - y0) - r < 0
bot_left_mask = np.hypot(x - x0, y + t_h - y0) - r < 0
bot_right_mask = np.hypot(x + t_w - x0, y + t_h - y0) - r < 0
mask = np.logical_and.reduce((top_left_mask, top_right_mask,
bot_left_mask, bot_right_mask))
mask = mask.astype(np.uint8)
cv2.imwrite('mask.png', mask*255)
Here's what the "fancy" mask looks like:
Seems about right. It cannot be circular because of the template shape. If I run clock2.jpg with this mask I get:
It works. No candy bars are identified. But I wish I could do it in fewer lines of code...
EDIT:
I've done some profiling. I ran 100 cycles of the "easy" way and the "accurate" way and calculated frames per second (fps):
easy way: 12.7 fps
accurate way: 7.8 fps
so there is some price to pay for making the mask with NumPy. These tests were done on a relatively powerful workstation. It could get uglier on more modest hardware...
Method 1: 'mask' image before cv2.matchTemplate
Just for kicks, I tried to make my own mask of the image that I pass to cv2.matchTemplate to see what kind of performance I can achieve. To be clear, this isn't a proper mask -- I set all of the pixels to ignore to one color (black or white). This is to get around the fact only TM_SQDIFF and TM_CORR_NORMED support a proper mask.
#Alexander Reynolds makes a very good point in the comments that some care must be taken if the template image (the thing we're trying to find) has lots of black or lots of white. For many problems, we will know a priori what the template looks like and we can specify a white background or black background.
I use cv2.multiply, which seems to be faster than numpy.multiply. cv2.multiply has the added advantage that it automatically clips the results to the range 0 to 255.
import numpy as np
import cv2
import time
img = cv2.imread('clock1.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
template = cv2.imread('target.jpg')
t_h, t_w = template.shape[0:2] # template height and width
mask_background = 'WHITE'
start_time = time.time()
for i in range(100): # do 100 cycles for timing
# find circle in gray image using Hough transform
circles = cv2.HoughCircles(gray, method = cv2.HOUGH_GRADIENT, dp = 1,
minDist = 150, param1 = 50, param2 = 70,
minRadius = 131, maxRadius = 200)
i = circles[0,0]
x0 = i[0]
y0 = i[1]
r = i[2]
# display circle on color image
cv2.circle(img,(x0, y0), r,(0,255,0),2)
if mask_background == 'BLACK': # black = 0, white = 255 on grayscale
mask = np.zeros(img.shape, dtype = np.uint8)
elif mask_background == 'WHITE':
mask = 255*np.ones(img.shape, dtype = np.uint8)
cv2.circle(mask, (x0, y0), r, color = (1,1,1), thickness = -1)
img2 = cv2.multiply(img, mask) # element wise multiplication
# values > 255 are truncated at 255
# do the template match
result = cv2.matchTemplate(img2, template, cv2.TM_CCOEFF_NORMED)
# call minMaxLoc
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)
# draw found rectangle on img
if max_val > 0.4:
cv2.rectangle(img, max_loc, (max_loc[0]+t_w, max_loc[1]+t_h), (0,255,0), 4)
fps = 100/(time.time()-start_time)
print('fps ', fps)
cv2.imwrite('output.jpg', img)
Profiling results:
BLACK background 12.3 fps
WHITE background 12.1 fps
Using this method has very little performance hit relative to 12.7 fps in original question. However, it has the drawback that it will still find templates that still stick over the edge a little bit. Depending on the exact nature of the problem, this may be acceptable in many applications.
Method 2: use cv2.boxFilter to create mask for minMaxLoc
In this technique, we start with a circular mask (as in OP), but then modify it with cv2.boxFilter. We change the anchor from default center of kernel to the top left corner (0, 0)
import numpy as np
import cv2
import time
img = cv2.imread('clock1.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
template = cv2.imread('target.jpg')
t_h, t_w = template.shape[0:2] # template height and width
print('t_h, t_w ', t_h, ' ', t_w)
start_time = time.time()
for i in range(100):
# find circle in gray image using Hough transform
circles = cv2.HoughCircles(gray, method = cv2.HOUGH_GRADIENT, dp = 1,
minDist = 150, param1 = 50, param2 = 70,
minRadius = 131, maxRadius = 200)
i = circles[0,0]
x0 = i[0]
y0 = i[1]
r = i[2]
# display circle on color image
cv2.circle(img,(x0, y0), r,(0,255,0),2)
# do the template match
result = cv2.matchTemplate(img, template, cv2.TM_CCOEFF_NORMED)
# finally, here is the part that gets tricky. we want to find highest
# rated match inside circle and we'd like to use minMaxLoc
# start to make mask by drawing circle on zero array
mask = np.zeros(result.shape, dtype = np.float)
cv2.circle(mask, (x0, y0), r, color = 1, thickness = -1)
mask = cv2.boxFilter(mask,
ddepth = -1,
ksize = (t_w, t_h),
anchor = (0,0),
normalize = True,
borderType = cv2.BORDER_ISOLATED)
# mask now contains values from zero to 1. we want to make anything
# less than 1 equal to zero
_, mask = cv2.threshold(mask, thresh = 0.9999,
maxval = 1.0, type = cv2.THRESH_BINARY)
mask = mask.astype(np.uint8)
# call minMaxLoc
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result, mask = mask)
# draw found rectangle on img
if max_val > 0.4:
cv2.rectangle(img, max_loc, (max_loc[0]+t_w, max_loc[1]+t_h), (0,255,0), 4)
fps = 100/(time.time()-start_time)
print('fps ', fps)
cv2.imwrite('output.jpg', img)
This code gives a mask identical to OP, but at 11.89 fps. This technique gives us more accuracy with slightly more performance hit than Method 1.