I have the following problem which is mainly algorithmic.
Let ABCD be a rectangle with known dimensions d1, d2 lying somewhere in space.
The rectangle ABCD is projected on a plane P (forming in the general case a trapezium KLMN). I know the projection matrix H.
I can also find the 2D coordinates of the trapezium edge points K,L,M,N.
The Question is the following :
Given the Projection Matrix H, The coordinates of the edges on the trapezium and the knowledge that our object is a rectangle with specified geometry (dimensions d1, d2), could we calculate the 3D coordinates of the points A, B, C, D ?
I am grabbing images of simple rectangles with a single camera and i want to reconstruct the rectangles on space. I could grab more than one image and use triangulation but this is not desired.
The projection Matrix alone isn't enough since a ray is projected to the same point. The fact that the object has known dimensions, makes me believe that the problem is solvable and there are finite solutions.
If I figure out how this reconstruction can be made I know how to program it. So I am asking for an algorithmic/math answer.
Any ideas are welcome
Thanks
You need to calculate the inverse of your projection matrix. (your matrix cannot be singular)
I'm going to give a fairly brief answer here, but I think you'll get my general drift. I'm assuming you have a 3x4 projection matrix (P), so you should be able to get the camera centre by finding the right null vector of P: call it C.
Once you have C, you'll be able to compute rays with the same direction as vectors CK,CL,CM and CN (i.e. the cross product of C and K,L,M or N, e.g. CxK)
Now all you have to do is compute 3 points (u1,u2,u3) which satisfies the following 6 constraints (arbitrarily assuming KL and KN are adjacent and ||KL|| >= ||KN|| if d1 >= d2):
u1 lies on CK, i.e. u1.CK = 0
u2 lies on CL
u3 lies on CN
||u1-u2|| = d1
||u1-u3|| = d2
(u1xu2).(u1xu3) = 0 (orthogonality)
where, A.B = dot product of vectors A and B
||A|| = euclidean norm of A
AxB = cross product of A and B
I think this problem will generate a set of possible solutions, at least in 2D it does. For the 2D case:
|
-----------+-----------
/|\
/ | \
/ | \
/---+---\VP
/ | \
/ | \
/ | \
/ | \
/ | -- \
/ | | \
/ | | \
In the above diagram, the vertical segment and the horizontal segment would project to the same line on the view plane (VP). If you drew this out to scale you'd see that there are two rays from the eye passing through each end point of the unprojected line. This line can be in many positions and rotations - imagine dropping a stick into a cone, it can get stuck in any number of positions.
So, in 2D space there are an infinite number of solutions within a well defined set.
Does this apply to 3D?
The algorithm would be along the lines of:
Invert the projection matrix
Calculate the four rays that pass through the vertices of the rectangle, effectively creating a skewed pyramid
Try and fit your rectangle into the pyramid. This is the tricky bit and I'm trying to mentally visualise rectangles in pyramids to see if they can fit in more than one way.
EDIT: If you knew the distance to the object it would become trivial.
EDIT V2:
OK, let Rn be the four rays in world space, i.e. transformed via the inverse matrix, expressed in terms of m.Rn, where |Rn| is one. The four points of the rectange are therefore:
P1 = aR1
P2 = bR2
P3 = cR3
P4 = dR4
where P1..P4 are the points around the circumference of the rectangle. From this, using a bit of vector maths, we can derive four equations:
|aR1 - bR2| = d1
|cR3 - dR4| = d1
|aR1 - cR3| = d2
|bR2 - dR4| = d2
where d1 and d2 are the lengths of the sides of the rectangle and a, b, c and d are the unknowns.
Now, there may be no solution to the above in which case you'd need to swap d1 with d2. You can expand each line to:
(a.R1x - b.R2x)2 + (a.R1y - b.R2y)2 + (a.R1z - b.R2z)2 = d12
where R1? and R2? are the x/y/z components of rays 1 and 2. Note that you're solving for a and b in the above, not x,y,z.
m_oLogin is right. If I understand your goal, the image the camera snaps is the plane P, right? If so, you're measuring K,L,M,N off the 2D image. You need the inverse of the projection matrix to reconstruct A,B,C, and D.
Now I've never done this before, but it ocurrs to me that you might run into the same problem GPS does with only 3 satellite fixes - there are two possible solutions, one 'behind' P and one 'in front' of it, right?
The projection matrix encapsulates both the perspective and scale, so the inverse will give you the solution you are after. I think you are assuming that it only encapsulates the perspective, and you need something else to choose the correct scale.
Related
I am working on an Eye Tracking application, and when I detect the pupil and enveloping it with an ellipse I have to compare it to a ground-truth (exact ellipse around the pupil).
There are always 3 cases of course:
No Overlap >> overlap = intersection = 0
Partial to Perfect Overlap >> overlap = intersection area / ground-truth area
Enclosing >> overlap = intersection area / ground truth
My problem is the 3rd case where e.g. found ellipse is much bigger than the ground-truth hence enclosing it inside so the total overlap is given as 1.0 which is mathematically right but detection-wise not really as the found ellipse doesn't only contain the pupil inside it but other non-pupil parts.
The question is:
What would be the best approach to measure and calculate the overlap percentage between the found and ground-truth ellipses? would be just mere division of the areas?
Please give some insights.
P.S.: I am coding with python and tried to use shapely library for the task as mentioned in the answer to this question as supposedly it does the transform to position the ellipses correctly regarding their rotational angle.
Let R be the reference ellipse, E the calculated ellipse.
We define score := area(E ∩ R) / area(E ∪ R). The larger the score the better the match.
As ∅ ⊆ E ∩ R ⊆ E ∪ R, we have 0 ≤ score ≤ 1, score=0 ⇔ (E ∩ R = ∅) and
score=1 ⇔ E=R.
Consider an ellipse that is completely enclosed by R and has half the area, as well as an ellipse that completely encloses R and has twice the area. Both would have a score of 0.5 . If they were closer to R, for example if the first had 4/5 the area and the second 5/4 the area both would have a score of 0.8 .
for back face culling algorithm I need to find the normal vector for each polygon.
given 3 points, I want to find the normal of a plane.
so I know how to do it :
find 2 vectors on the plane
find their cross vector - which will give me the normal vector (a,b,c)
my question is, does it matter what is the order of the points when I find 2 vectors?
for ex: given 3 points: p1(0,0,0), p2(5,0,0), p3(10,10,10)
does it matter if I choose vector
V1=(p2-p1)=(5, 0, 0)-(0, 0, 0)=(5, 0, 0)
V2=(p3-p1)=(10,10,10)-(0, 0, 0)=(10, 10, 10)
or
v1=(p1-p2)
v2=(p1-p3)
your polygon has vertexes a, b, c.
you calculate the vectors:
v1 = a-c
v2 = b-c
this refers a and b to c. It would be the same if you decided to refer, say, b and c to a.
calculate the cross product v1*v2 (this gives a vector perpendicular to v1 and v2) and normalize it.
If, you did calculate (a-b)(a-c) instead of (b-a)(c-a), the resulting vector will be mirrored (ie, pointing to the wrong direction).
OT: normalize with/see http://en.wikipedia.org/wiki/Fast_inverse_square_root that was developed exactly to calculate face normals
Given a convex polygon P and a point A on P's boundary, how do I compute a point B also on P's boundary such that AB splits P into two areas of a given proportion?
Ideally I'd like an analytical solution. As a last resort I can draw a line anywhere on the polygon and gradually move it until the proportion is correct to a given precision.
I've worked out how to calculate B once I know between which two points on the polygon it should go. So if there's a way to find out between which points it should go, I should be able to take it from there!
Split the polygon into triangles from the point A, and calculate their areas. Then you can add triangles from each end to each polygon depending on their proportions, until there is only one triangle left. Then you know that the point B is somewhere on the base of that triangle.
As often is the case, I've answered my own question only minutes after posting it!
My code to determine between which points B should go looks something like:
while areaSoFar + areas[i] < targetArea:
i++
areaSoFar += areas[i]
It turns out that I just needed to insert the last element of the area summation formula into the same check:
while areaSoFar + areas[i] + points[i].x * start.y - points[i].y * start.x < targetArea:
i++
areaSoFar += areas[i]
Note that the areas[] array above contains each element of the area summation formula.
This is similar in spirit to Guffa's answer, but slightly more efficient.
How can I calculate the minimum distance between two rectangles?It is easy for rectangles which have no angles (i.e. 0 degrees one), but for rotated rectangles with any different angles I do not know how to do it.
Can you recommend any way?
WhiteFlare
Check either they intersect first (try to take point from one rectangle and check either it is inside other rectangle).There are several ways to do it. One method (not the best one, but easy to explain) is the following. Let A1, A2, A3, A4 - rectangle points, T - some other point. Then count squares for triangles: S1 = (A1,A2,T), S2 = S(A2,A3,T), S3 = S(A3, A4, T), S4 = S(A4, A1, A2). Let S_rectangle be reactangle square. Then T lies inside rectangle <=> S1 + S2 + S3 + S4 = S_rectangle.
If reactangles don't intersect each other, then do these steps.
Calculate coordinates of all 8 points of 2 rectangles.
Take minimum among all 4 * 4 = 16 pairs of points (points from different rectangles). Let's denote it min_1.
Then, take some point from the first rectangle (4 ways to do it), take 4 segments of another rectangle (4 ways), check either perpendicular from that point to that segment gets inside segment. Take the mininmum of such perpendiculars. Let's denote it min_2.
The same as in 3, but take point from the second rectangle, lines from the first: you get min_3.
result = min(min_1, min_2, min_3)
Calculate coordinates of all 8
points of 2 rectangles.
Take the two lowest distances among
all 4 * 4 = 16 pairs of points
(points from different rectangles).
And get the 3 points P1, P2 and P3
{Two of them belong to one rectangle
and the third to the other}
The 2 Points belong to one rectangle
should considered as segment, Now
find the Short distance between a
segment and the third point.
I actually have two questions, I found the answer to the second and didn't update the diagram. I'm not actually sure if these are possible, they really stumped me.
Question 1:
Given point A and e, the angle of the line A is on relative to the x-axis where 0<=e<360 degrees, how do you calculate the coordinates of B? BA is perpendicular to A's line and 1 unit long.
SOLVED: I start by taking the unit vector from a parallel to the x-axis and then I rotate it 90 + e degrees.
Question 2:
I'm using this approach. If anyone has any better suggestions, please let me know.
SOLVED: I find the dot product of the vector from step 1 and the normalized vector AC.
Question 3:
This one should be pretty self-explanatory from the diagram. I need to find the coordinates of C given A, B, the angle of BAC and the distance between A and C.
SOLVED: I rotate BA e degrees and then change the magnitude to d.
If anyone spots problems with my solutions, please comment.
Easy if you understand vectors. Learn about how 2D vectors work and you'll have it. Is that the course you're taking?
Take the unit vector from e to A, knowing that a unit vector has length 1. Assume l1 = xi + yj. The perpendicular vector has components that are the reverse of l1 with one sign changed. In this case, l2 = -yi + xj.
Take the vector l2 that you got from the first problem and transform it as follows:
cx = -cos(t)y - sin(t)x
cy = +sin(t)y + cos(t)x
where t is the rotation angle in radians.
I'll leave the third one for you. Read about 2D vectors and transformations.