Given an input of 2D points, I would like to segment them in lines. So if you draw a zig-zag style line, each of the segments should be recognized as a line. Usually, I would use OpenCV's
cvHoughLines or a similar approach (PCA with an outlier remover), but in this case the program is not allowed to make "false-positive" errors. If the user draws a line and it's not recognized - it's ok, but if the user draws a curcle and it comes out as a square - it's not ok. So I have an upper bound on the error - but if it's a long line and some of the points have a greater distance from the approximated line, it's ok again. Summed up:
-line detection
-no false positives
-bounded, dynamically adjusting error
Oh, and the points are drawn in sequence, just like hand drawing.
At least it does not have to be fast. It's for a sketching tool. Anyone has an idea?
This has the same difficulty as voice and gesture recognition. In other words, you can never be 100% sure that you've found all the corners/junctions, and among those you've found you can never be 100% sure they are correct. The reason you can't be absolutely sure is because of ambiguity. The user might have made a single stroke, intending to create two lines that meet at a right angle. But if they did it quickly, the 'corner' might have been quite round, so it wouldn't be detected.
So you will never be able to avoid false positives. The best you can do is mitigate them by exploring several possible segmentations, and using contextual information to decide which is the most likely.
There are lots of papers on sketch segmentation every year. This seems like a very basic thing to solve, but it is still an open topic. The one I use is out of Texas A&M, called MergeCF. It is nicely summarized in this paper: http://srlweb.cs.tamu.edu/srlng_media/content/objects/object-1246390659-1e1d2af6b25a2ba175670f9cb2e989fe/mergeCF-sbim09-fin.pdf.
Basically, you find the areas that have high curvature (higher than some fraction of the mean curvature) and slow speed (so you need timestamps). Combining curvature and speed improves the initial fit quite a lot. That will give you clusters of points, which you reduce to a single point in some way (e.g. the one closest to the middle of the cluster, or the one with the highest curvature, etc.). This is an 'over fit' of the stroke, however. The next stage of the algorithm is to iteratively pick the smallest segment, and see what would happen if it is merged with one of its neighboring segments. If merging doesn't increase the overall error too much, you remove the point separating the two segments. Rinse, repeat, until you're done.
It has been a while since I've looked at the new segmenters, but I don't think there have been any breakthroughs.
In my implementation I use curvature median rather than mean in my initial threshold, which seems to give me better results. My heavily modified implementation is here, which is definitely not a self-contained thing, but it might give you some insight. http://code.google.com/p/pen-ui/source/browse/trunk/thesis-code/src/org/six11/sf/CornerFinder.java
Related
I'm trying to find powerlines in LIDAR points clouds with skimage.measures ransac() function. This is my very first time meddling with these modules in python so bear with me.
So far all I knew how to do reliably was filtering low or 'ground' points from the cloud to reduce the number of points to deal with.
def filter_Z(las, threshold):
filtered = laspy.create(point_format = las.header.point_format, file_version = las.header.version)
filtered.points = las.points[las.Z > las.Z.min() + threshold]
print(f'original size: {len(las.points)}')
print(f'filtered size: {len(filtered.points)}')
filtered.write('filtered_points2.las')
return filtered
The threshold is something I put in by hand since in the las files I worked with are some nasty outliers that prevent me from dynamically calculating it.
The filtered point cloud, or one of them atleast looks like this:
Note the evil red outliers on top, maybe they're birds or something. Along with them are trees and roofs of buildings. If anyone wants to take a look at the .las files, let me know. I can't put a wetransfer link in the body of the question.
A top down view:
I've looked into it as much as I could, and found the skimage.measure module and the ransac function that comes with it. I played around a bit to get a feel for it and currently I'm stumped on how to continue.
def ransac_linefit_sklearn(points):
model_robust, inliers = ransac(points, LineModelND, min_samples=2, residual_threshold=1000, max_trials=1000)
return model_robust, inliers
The result is quite predictable (I ran ransac on a 2D view of the cloud just to make it a bit easier on the pc)
Using this doesn't really yield any good results in examples like the one I posted. The vegetation clusters have too many points and the line is fitted through it because it has the highest point density.
I tried DBSCAN() to cluster up the points but it didn't work. I also attempted OPTICS() but as I write it still hasn't finished running.
From what I've read on various articles, the best course of action would be to cluster up the points and perform RANSAC on each individual cluster to find lines, but I'm not really sure on how to do that or what clustering method to use in situations like these.
One thing I'm also curious about doing is just filtering out the big blobs of trees that mess with model fititng.
Inadequacy of RANSAC
RANSAC works best whenever your data fits a mono-modal distribution around your model. In the case of this point cloud, it works best whenever there is only one line with outliers, but there are at least 5 lines when viewed birds-eye. Check out this older SO post that discusses your problem. Francesco's response suggests an iterative RANSAC based approach.
Octrees and SVD
Colleagues worked on a similar problem in my previous job. I am not fluent in the approach, but I know enough to provide some hints.
Their approach resembled Francesco's suggestion. They partitioned the point-cloud into octrees and calculated the singular value decomposition (SVD) within each partition. The three resulting singular values will correspond to the geometric distribution of the data.
If the first singular value is significantly greater than the other two, then the points are line-like.
If the first and second singular values are significantly greater than the other, then the points are plane-like
If all three values are of similar magnitude, then the data is just a "glob" of points.
They used these rules iteratively to rule out which points were most likely NOT part of the lines.
Literature
If you want to look into published methods, maybe this paper is a good starting point. Power lines are modeled as hyperbolic functions.
I am sketching out a new simulation that will involve thousands of ships moving around on Earth's oceans and interacting over long periods of time. So, lots of "intersection detection" for sensor and communications ranges, as well as region detection for various environmental conditions. We'll assume a spherical earth, not WGS84. This is an event-step simulation that spits out metrics, not a real time game or anything like that.
A question is to use Cartesian coordinates (Earth-Centered, Earth-Fixed) or Geodic/polar coordinates. With polar coordinates a ship's track would be internally represented as a series of lat/lon waypoints with times and a great circle paths between them. With a Cartesian representation the waypoints would be connected with polyline renderings of the great circle between them.
The reason this is a question is I suspect that by sticking to a Cartesian data model it becomes possible to use various geometry libraries that are performance tuned, and even offer up SIMD/GPU performance advantages. The polar coordinates would probably be the more natural way to proceed if writing everything from scratch. But I suspect that by keeping things Cartesian I will have greater access to better and faster libraries. Is this an invalid line of thought? Another consideration is that I know polar coordinate calculations tend to get really screwy when near the poles.
Just curious if somebody with experience could save me a whole lot of time prototyping some scenarios both ways.
It often works well to represent directions as unit vectors instead of angles. Rotation of a vector by another angle becomes a 2x2 or 3x3 matmul (efficient with SIMD, but still more expensive than an FP add of two numbers in radians), but you very rarely need sin/cos.
You may occasionally want atan2 to get an angle, but usually not inside tight loops.
Intersection-detection can be very efficient (with SIMD) for XYZ coordinates given another XYZ + range. I'm not sure how efficiently you could check which lat/lon pairs were within range of a given point, not a problem I've looked at.
IDK what kind of stuff you'd find in existing libraries, or what you'd want to do with it.
I am working on a graphic application in which the user draws lines.
The lines are not smooth. I see the problem especially when drawing the lines slowly.
What would be the correct way to smoothen the lines?
I read the following great article: http://www.merowing.info/2012/04/drawing-smooth-lines-with-cocos2d-ios-inspired-by-paper/
It fixed some of the problem, but not all of it. Still, when drawing lines slowly, they look jagged:
The problem is actually that you have too many sample points.
This is why the problem is exacerbated when the user inputs points slowly, rather than quickly. One possible solution is to use some sort of best-fitting line equation (e.g. Least Squares) to reduce the number of points and simplify it to an approximation.
There are more sophisticated approaches to smoothing user input, but in this simple case all you really need to do is reduce the sample size.
Suppose I have a QR code, how would I look for the timing information (and therefore determining the version and size of the code)?
When I detect a code, I look for the three dark squares, the alignment information. I can then morph the picture, so that the picture is upright, with all edges and modules the same size and in the correct orientation. The next step is to look for timing information, so I can start decoding the code. But how do I find it? I don't know how many modules there are in one line at that point, and therefore, I don't know the exact location. How would I detect the timing lines?
Please note, that I can't use a library to decode the picture information. The idea is to recognize the picture information.
There are many ways you could estimate the module size.
ZXing estimates it based on the 1-1-3-1-1 ring pattern of the finders.
There's actually a set timing pattern between the finders, if you know their positions fairly accurately. It's just a matter of walking the pixels and calculating the average length of the white and black regions.
The ZXing estimate is a bit cruder but works pretty well. There are cases where it produces an incorrect estimate that walking the timer pattern would probably eliminate. But the common use case for ZXing is a live stream of images and one incorrect estimate isn't a big deal.
Crayon Physics Deluxe is a commercial game that came out recently. Watch the video on the main link to get an idea of what I'm talking about.
It allows you to draw shapes and have them react with proper physics. The goal is to move a ball to a star across the screen using contraptions and shapes you build.
While the game is basically a wrapper for the popular Box2D Physics Engine, it does have one feature that I'm curious about how it is implemented.
Its drawing looks very much like a Crayon. You can see the texture of the crayon and as it draws it varies in thickness and darkness just like an actual crayon drawing would look like.
(source: kloonigames.com)
(source: kloonigames.com)
The background texture is freely available here.
What kind of algorithm would be used to render those lines in a way that looks like a Crayon? Is it a simple texture applied with a random thickness and darkness or is there something more going on?
I remember reading (a long time ago) a short description of an algorithm to do so:
for the general form of the line, you split the segment in two at a random point, and move this point slightly away from it's position (the variation depending on the distance of the point to the extremity). Repeat recursively/randomly. In this way, you lines are not "perfect" (straight line)
for a given segment you can "overshoot" a little bit, by extending one extremity or the other (or both). In this way, you don't have perfect joints. If i remember well, the best was to extends the original extremities, but you can do this for the sub-segment if you want to visibly split them.
draw the lines with pattern/stamp
there was also the (already mentioned) possibility to drawn with different starting and ending opacity (to mimic the tendency to release the pen at the end of drawing)
You can use a different size for the stamp on the beginning and the end of the line (also to mimic the tendency to release the pen at the end of drawing). For the same effect, you can also draw the line twice, with a small variation for one of the extremity (be careful with the alpha in this case, as the line will be drawn twice)
Last, for a given line, you can do the previous modifications several times (ie draw the line twice, with different variations) : human tend to repeat a line if they make some mistakes.
Regards
If you blow the image up you can see a repeating stamp-pattern .. there's probably a small assortment that it uses as it moves from a to b - might even rotate them ..
The wavering of the line can't be all that difficult to do. Divide into a bunch of random segments, pick positions slightly away from the direct route and draw splines.
Here's a paper that uses a lot of math to simulate the deposition of wax on paper using a model of friction. But I think your best bet is to just use a repeating pattern, as another reader mentioned, and vary the opacity according to pressure.
For the imperfect line drawing parts, I have a blog entry describing how to do it using bezier curves.
You can base darkness on speed. That's just measuring the distance traveled by the cursor between this frame and the last frame (remember Pythagorean theorem) and then when you go to draw that line segment on screen, adjust the alpha (opacity) according to the distance you measured.
There is a paper available called Mimicking Hand Drawn Pencil Lines which covers a bit of what you are after. Although it doesn't present a very detailed view of the algorithm, the authors do cover the basics of the steps that they used.
This paper includes high level descriptions of how they generated the lines, as well as how they generated the textures for the lines, and they get results which are similar to what you want.
This article on rendering chart series to look like XKCD comics has an algorithm for perturbing lines which may be relevant. It doesn't cover calculating the texture of a crayon drawn line, but it does offer an approach to making a straight line look imperfect in a human-like way.
Example output:
I believe the easiest way would simply be to use a texture with random darkness (some gradients, maybe) throughout, and set size randomly.