I have a image with horizontal and vertical lines. In fact, this image is the BBC website converted to horizontal and vertical lines.
My problem is that I want to be able to find all the rectangles in the image. I want to write a computer program to find all the rectangles.
Does anyone know how to do this or suggest ideas on how to get started? This task is easy for me as a person to find the visual rectangles, but I am not sure how to describe it as a program.
Image is the BBC website here http://www.bbc.co.uk/
Update to this, I wrote the code which converts the BBC website image to the horizontal and vertical line, the problem is these lines do not completely meet at the corners and sometimes they do not completely form a rectangle. Thanks!
Opencv (image processing and computer vision library written in c) has implementation for hough transform (the simple hough transform find lines in an image, while the generalized one finds more complex objects) so that could be a good start. For the rectangles which do have closed corners there are also corner detectors such as cornerHarris which can help.
I ran the houghlines demo provided with opencv and here's the result on the image you gave (detected lines marked in red):
(source: splintec.com)
I believe you are looking for the generalized Hough transform.
In computer vision there is a algorithm called Generalized Hough Transform which maybe can solve your problem. There should be open source code having implemented this algorithm. Just search for it.
Assuming it's a reasonably noise free image (not a video of a screen) then one of the simple floodfill algorithms should work. You might need to run a dilate/erode on the image to close up the gaps.
The normal way to find lines is a Hough transform ( then find lines at right angles)
Opencv is the easiest way.
Take a look at this question OpenCV Object Detection - Center Point
There are several different approaches to your problem. I'd use a morphological image processing tool like this one. You will have the flexibility to define "rectangle" even something that not "exactly closed" (where the fill algorithm will fail).
Another possibility could be to use a machine learning approach, which basically is more data-driven than definition-driven like the previous one. You'll have to give your algorithm several "examples" of what a rectangle is, and it will eventually learn (with a bias and an error rate).
iterate from left to right until you hit a color pixel then use modified flood fill algorithm. more info on the algo flood fill # wiki
another approach would be to find ANY colored pixel on the image then go with
while(pixel under current is colored)
{
lowest pixel coordinate = pixel under current
current = pixel under
}
then do the same upwards.
now u have defined a single line. then use ends of the lines to approx match lines into rectangles. if they are not pixel perfect you could do some kind of tresholding.
The flood fill would work, or you could use a modification of an edge tracking algorithm.
what you do is:
create a 2d array (or any other d2 data struct)- each row represents a horizontal pixel line on screen, and each column a vertical line
iterate through all the pixels, left to right, and whenever you find a coloured one add its coordinates to the array
iterate through the array and findying lines and storing the begin and end pixel for each one (different data structure)
knowing that the begin of each line is its left/top pixel, you can easily check to see if any 4 lines comprise a rectangle
To get from the image you have with the nearly touching horizontal and vertical lines to just the rectangles:
Convert to binary (i.e. all lines
are white, the rest is black)
Perform a Binary dilation (here you make every pixel that touches a white pixel in the source image or is a white pixel in the source image white. Touch is straight only (so each pixel "touches" the pixels to its left, right, above and below it) this is called "4-connected"
repeat step 3 a few times if the gaps between the ends are larger then 2 pixels wide, but not too often!
Perform a skeleton operation (here you make every pixel in the output image black if it is a white pixel in the source image that touches at least one black pixel and the white pixels it touches (in the source image) all touch eachother. Again touch defined with 4-connectedness. See sample below.
Repeat step 4 untill the image doesn't change after a repeat (all white pixels are line ends or connectors)
This will, with a bit of luck, first show the boxes with thick fat lines, leaving thick fat artifacts all over the image (after step 3) and then then after step 5 all thick fat artifacts will have been removed, while all boxes remain. You need to tweek the number of repeats in step 3 for best results. If you're interested in image morphology, this is the book of a really good introductory course I took.
Sample: (0=black, 1=white, pixels in the center of each 3x3 block are being considered, input left, output right)
011 => 011
011 => 001 all other white pixels touch, so eliminate
011 => 011
010 => 010
010 => 010 top pixel would become disconnected, so leave
010 => 010
010 => 010
010 => 000 touches only one white pixel, so remove
000 => 000
010 => 010
111 => 111 does not touch black pixels, leave
010 => 010
010 => 010
011 => 011 other pixels do not touch. so leave
000 => 000
Related
Problem
I'm trying to use opencv2 to detect PlayStation Move Motion Controllers in still images. In an attempt to increase the contrast between the orbs and the backgrounds, I decided to modify the input image to automatically scale the brightness level between the image's mean level and 96 above for each channel, then when converting to grayscale, taking the maximum value instead of the default transform, since some orbs are saturated but not "bright".
However, my best attempts at adjusting the parameters seems to not work well, detecting circles that aren't there over the obvious ones.
What can I do to improve the accuracy of the detection? What other improvements or algorithms do you think I could use?
Samples
In order of best to worst:
2 Wands, 1 Wand detected (showing all 2 detected circles)
2 Wands, 1 Wand detected with many nonexistent circles (showing top 4 circles)
1 Wand (against a dark background), 6 total circles, the lowest-ranked of which is the correct one (showing all 6 circles)
1 Wand (against a dark background), 44 total circles detected, none of which are that Wand (showing all 44 circles)
I am using this function call:
cv2.HoughCircles(img_gray,cv2.HOUGH_GRADIENT,
dp=1, minDist=24, param1=90, param2=25,
minRadius=2, maxRadius=48)
All images are resized and cropped to 640x480 (the resolution of the PS3 Eye). No blur is performed.
I think hough circles is the wrong approach for you, as you are not really looking for circles. You are looking for circular areas with strong intensity. Use e.g. blob detection instead, I linked a guide:
https://www.learnopencv.com/blob-detection-using-opencv-python-c/
In the blob detection, you need to set the parameters to get a proper high-intensity circular area.
as the other user said, hough circles arent the best approach here because hough circles look for perfect circles only. whereas your target is "circular" but not a circle (due to motion blur, light bleed/reflection, noise etc)
I suggest converting the image to HSV then filtering by hue/color and intensities to get a binary threshold instead of using grayscale directly (that will help remove background & noise and limit the search area)
then using findContours() (faster than blob detection), check for contours of high circularity and expected size/area range and maybe even solidity.
area = cv2.contourArea(contour)
perimeter = cv2.arcLength(contour,True)
circularity = 4*np.pi*area / (perimeter**2)
solidity = area/cv2.contourArea(cv2.convexHull(contour))
your biggest problem will be the orb contour merging with the background due to low contrast. so maybe some adaptive threshold could help
I need to find the shoulder points(humans body) of the image need help how to identify the coordinates, I am able to get the edges of the human body using imagemagick but having difficulties in finding Shoulder points A,B and neck point C.
Original Image
Image after getting edges
Scan your edge image row by row from top to bottom. For each row calculate the length between the first edge pixel to the last.
The row were you have a big change in the length is point C.
After you found point C the length start to go up in a linear way with some slope m when the slope changes very drastically is when you encounter the row with the shoulder points. You may need to calculate this slope seperattly for each side.
Try processing the input image with Image Magick morphology methods. Detailed examples are here. The script below will size down and threshold the image to 2bpp bitmap, then it will try to close all the gaps, and round the shape. Afterwards a skeleton is created that holds general information about our shape. Finally using hit and miss kernel you search for specific line ends. In this case diagonal ones. The output is an image with set of dots. However you can use Image Magick's identify to print pixel positions. I also assume that more or less you know where shoulders are and can discard points in other parts of the image. The C-point, identifying neck can be extrapolated from A and B.
convert -resize 25% input.jpg -negate -normalize -colorspace gray -threshold 20% -trim +dither v.png
convert v.png -morphology Close Disk x.png
convert x.png -morphology Thinning:-1 Skeleton:2 y.png
convert y.png -morphology HMT 'LineEnds:2;LineEnds:2>>' z.png
I used IM v.6.7.9 as I have some issues with v7.
I need to be able to turn a black and white image into series of lines (start, end points) and circles (start point, radius). I have a "pen width" that's constant.
(I'm working with a screen that can only work with this kind of graphics).
Problem is, I don't want to over complicate things - I could represent any image with loads of small lines, but it would take a lot of time to draw, so I basically want to "approximate" the image using those lines and circles.
I've tried several approaches (guessing lines, working area by area, etc) but none had any reasonable results without using a lot of lines and circles.
Any idea on how to approach this problem?
Thanks in advance!
You don't specify what language you are working in here but I'd suggest OpenCV if possible. If not, then most decent CV libraries ought to support the features that I'm about to describe here.
You don't say if the input is already composed of simple shapes ( lines and polygons) or not. Assuming that it's not, i.e. it's a photo or frame from a video for example, you'll need to do some edge extraction to find the lines that you are going to model. Use a Canny or other edge detector to convert the image into a series of lines.
I suggest that you then extract Circles as they are the richest feature that you can model directly. You should consider using a Hough Circle transform to locate circles in your edge image. Once you've located them you need to remove them from the edge image (to avoid duplicating them in the line processing section below).
Now, for each pixel in the edge image that's 'on' you want to find the longest line segment that it's a part of. There are a number of algorithms for doing this, simplest would be Probabilistic Hough Transform (also available in openCV) to extract line segments which will give you control over the minimum length, allowed gaps etc. You may also want to examine alternatives like LSWMS which has OpenCV source code freely available.
Once you have extracted the lines and circles you can plot them into a new image or save the coordinates for your output device.
After running Canny edge detector on an image i'm getting clear lines. But the Hough line function seems to be missing out on pretty prominent lines when run on the Canny edgemap image.
I'm keeping only vertical and horizontal Hough lines (a tolerance of 15 degrees). Lots of extra lines are coming up but clearly visible lines bounding the rectangles are not being picked up.
Here's the snippet:
cvCanny( img, canny, 0, 100, 3 );
lines = cvHoughLines2( canny, storage, CV_HOUGH_PROBABILISTIC, 1, CV_PI/180, 35, 20, 10 );
The main intention is to detect the rectangular boxes that denote the nodes of the linked list. However the squares.c sample program will detect only perfect rectangles, not if an arrowhead is touching the rectangle boundary.
Could you please explain the sort of changes to Hough line function which will help me get hough lines corresponding to clearly visible lines in Canny edge image?
(Added: a preprocessing step, suggested by shernshiou.)
Preprocessing steps:
Thresholding the image,
Use connected-component
From the connected-component results, detect and remove the small objects - the sets of four-digits below and in the middle of each box.
(Remark. The thresholding step is simply a preprocessing step required by connected-component.)
If you want to detect only perfectly horizontal and vertical lines, my suggestion is to perform horizontal and vertical edge enhancement (via convolution) before Hough transform.
This will make the true lines more likely to "peak" in the Hough-projection, and increases the chance of the line being picked up by OpenCV.
The steps would be:
Compute Canny edge image from input
Apply horizontal Sobel filtering on Canny edge image
Apply Hough line detection on horizontally-enhanced edge image.
Apply vertical Sobel filtering on Canny edge image. (Note: use step 1's result, not step 2's)
Apply Hough line detection on vertically-enhanced edge image.
Combine the horizontal and vertical lines and present the result.
You did read the documentation did you?
I have a few options for you:
The lines you miss (most notably the leftmost vertical line on the rightmost box in the image) are rather short. Try lowering the threshold (5th input variable of cvHoughLines2). This threshold is just the number of pixels that must lie on the line. From the image I'd guess that there are indeed less than 35 pixels on the lines you miss.
The 6th input variable indicates the minimum line length. I assume this is in pixels, so with the 5th parameter you require 35 pixels on the line, yet you search for lines 20 pixels or longer. The way you set this variable it is non-functional. Lower the 5th variable, raise this one if you are finding to many useless short lines.
Lower the 7th parameter to disallow large gaps in your lines. This will eliminate some of the slanted lines.
In short, try it again with different values for parameters 5,6 and 7.
I'd try some lower values of parameters 5 and 7, and a similar or slightly higher value for 6. Because of 2 above 5 should always be lower than or equal to 6 to have an effect. 7 should at least equal the difference between 5 and 6 if 5 is lower.
Normally people do not use hough line straight away from the box. The normal practice involves pre-processing image (e.g. change the luminance, change the colour, sharpen the image...).
I'm doing some graphics processing and I have a logic where in I have a bitmap with edges and I disregard all table edges from the letters E.g.
0000000000
0111111110
0100000010
0102220010
0100200010
0100200010
0100000010
0111111110
0000000000
0 - background color
1 - ignored edges
2 - edges I need
My logic is just simple, if a number of continuous pixels exceeds a certain threshold, e.g. 20pixels of continuous edges, it will consider it as a line and disregard it.
My problem is that on big font size and letters such as H and T, it will definitely exceed the threshold. Please advise is there a better way or additional logic i need to implement in order to separate table lines from letters.
[update] Additional consideration: Performance, this logic will be used during touch movement (dragging). It will be called a lot of times so it needs to be fast.
If table lines are guaranteed to be thin, then ignore thick lines. However, if the lines in your application are generated by edge detection (which are always 1-pixel thin) then connected-component will be needed.
Basically, the "thickness" refers to thickness measured from an edge profile:
00000000100000000 This line has thickness 1
00000011111000000 This line has thickness 5. However, this cannot occur in the output of edge detection, because edge detection algorithms are specifically designed to remove this condition.
00000000111111111 This is a transition from black to white.
Table lines usually have small thickness. Large fonts usually have transition from black to white because their thickness is larger than the edge profile window.