detecting lane lines on a binary mask - python-3.x

Question

I have a binary mask of a road, the mask is a little irregular(sometimes even more than depicted in the image).
I have tried houghLine in OpenCV to detect boundary lines, but the boundary lines are not as expected. I tried erosion and dilation to smooth out things, but no luck. Also since the path is curved it becomes even difficult to detect boundary lines using houghLines. How can I modify the code to detect lines better?
img2=cv2.erode(img2,None,iterations=2)
img2=cv2.dilate(img2,None,iterations=2)
can=cv2.Canny(img2,150,50)
lines=cv2.HoughLinesP(can,1,np.pi/180,50,maxLineGap=50,minLineLength=10)
if(lines is not None):
for x in lines:
#print(lines[0])
#mask=np.zeros(frame2.shape,dtype=np.uint8)
#roi=lines
#cv2.fillPoly(mask,roi,(255,255,255))
#cv2.imshow(mask)
for x1,y1,x2,y2 in x:
cv2.line(frame2,(x1,y1),(x2,y2),(255,0,0),2)

Answer 1

You say that Hough is failing but you don't say why. Why is your output "not as expected"? In my experience, Hough Line Detection’s critical points are two: 1) The edges mask you pass to it and 2) how you filter the resulting lines. You should be fine-tuning those two steps and Hough should be enough for your problem.
I don't know what kind of problems the line detector is giving you, but suppose you are interested (as your question suggests) in other methods for lane detection. There are at least two things you could try: 1) Bird's eye transform of the road – which makes line detection much easier since all your lines are now parallel lines. And 2) Contour detection (instead of lines).
Let's examine 2 and what kind of results you can obtain. Listen, man, I offer my answer in C++, but I make notes along with it. I try to highlight the important ideas, so you can implement them in your language of choice. However, if all you want is a CTRL+C and CTRL+V solution, that's ok, but this answer won't help you.
Ok, let's start by reading the image and converting it to binary. Our goal here is to first obtain the edges. Pretty standard stuff:
//Read input image:
std::string imagePath = "C://opencvImages//lanesMask.png";
cv::Mat testImage = cv::imread( imagePath );
//Convert BGR to Gray:
cv::Mat grayImage;
cv::cvtColor( testImage, grayImage, cv::COLOR_RGB2GRAY );
//Get binary image via Otsu:
cv::Mat binaryImage;
cv::threshold( grayImage, binaryImage, 0, 255, cv::THRESH_OTSU );
Now, simply pass this image to Canny's Edge detector. The parameters are also pretty standard. As per Canny's documentation, the ratios between lower and upper thresholds are related by a factor of 3:
//Get Edges via Canny:
cv::Mat testEdges;
//Setup lower and upper thresholds for edge detection:
float lowerThreshold = 30;
float upperThreshold = 3 * lowerThreshold;
cv::Canny( binaryImage, testEdges, lowerThreshold, upperThreshold );
Your mask is pretty good; these are the edges Canny finds:
Now, here's where we are trying something different. We won't use Hough's line detection, instead, let's find the contours of the mask. Each contour is made of points. What we are looking for is actually lines, straight lines that can be fitted to these points. There's more than a method for achieving that. I propose K-means, a clustering algorithm.
The idea is that the points, as you can see, can be clustered in 4 groups: The vanishing point of the lanes (those should be 2 endpoints there) and the 2 starting points of the road. If we give K-means the points of the contour and tell it to cluster the data in 4 separate groups, we should get the means (location) of those 4 points.
Let's try it out. The first step is to find the contours in the edges mask:
//Get contours:
std::vector< std::vector<cv::Point> > contours;
std::vector< cv::Vec4i > hierarchy;
cv::findContours( testEdges, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, cv::Point(0, 0) );
K-means needs a specific data type on its input. I'll use a cv::Point2f vector to store all the contour points. Let's set up the variables used by K-means:
//Set up the data containers used by K-means:
cv::Mat centers; cv::Mat labels;
std::vector<cv::Point2f> points; //the data for clustering is stored here
Next, let's loop through the contours and store each point inside the Point2f vector, so we can further pass it to K-means. Let’s use the loop to also draw the contours and make sure we are not messing things up:
//Loop thru the found contours:
for( int i = 0; i < (int)contours.size(); i++ ){
//Set a color & draw contours:
cv::Scalar color = cv::Scalar( 0, 256, 0 );
cv::drawContours( testImage, contours, i, color, 2, 8, hierarchy, 0, cv::Point() );
//This is the current vector of points that is being processed:
std::vector<cv::Point> currentVecPoint = contours[i];
//Loop thru it and store each point as a float point inside a plain vector:
for(int k = 0; k < (int)currentVecPoint.size(); k++){
cv::Point currentPoint = currentVecPoint[k];
//Push (store) the point into the vector:
points.push_back( currentPoint );
}
}
These are the contours found:
There, now, I have the contour points in my vector. Let's pass the info on to K-means:
//Setup K-means:
int clusterCount = 4; //Number of clusters to split the set by
int attempts = 5; //Number of times the algorithm is executed using different initial labels
int flags = cv::KMEANS_PP_CENTERS;
cv::TermCriteria criteria = cv::TermCriteria( CV_TERMCRIT_ITER | CV_TERMCRIT_EPS, 10, 0.01 );
//The call to kmeans:
cv::kmeans( points, clusterCount, labels, criteria, attempts, flags, centers );
And that's all. The result of K-means is in the centers matrix. Each row of the matrix should have 2 columns, denoting a point center. In this case, the matrix is of size 4 x 2. Let's draw that info:
As expected, 4 center points, each is the mean of a cluster. Very cool, now, is this approximation enough for your application? Only you know that! You could work with those points and extend both lines, but that's a possible improvement of this result.