I'm trying to find a match between a set of 2D boxes with coordinates (A) (from a template with known sizes and distances between boxes) to another set of 2D boxes with coordinates (B) (which may contain more boxes than A). They should match in terms of each box from A corresponds to a single Box in B. The boxes in A together form a "stamp" which is assymmetrical in atleast one dimension.
Illustration of problem
explanation: "Stanz" in the illustration is a box from set A.
One might even think of the Set A as only 2D points (the centerpoint of the box) to make it simpler.
The end result will be to know which A box corresponds to which B box.
I can only think of very specific ways of doing this, tailored to a specific layout of boxes, is there any known generic ways of dealing with this forms of matching/search problems and what are they called?
Edit: Possible solution
I have come up with one possible solution, looking for all the possible rotations at each possible B center position for a single box from set A. Here all of the points in A would be rotated and compared against the distance to B centers. Not sure if this is a good way.
Looking for the possible rotations at each B centerpoint- solution
In your example, the transformation between the template and its presence in B can be entirely defined (actually, over-defined) by two matching points.
So here's a simple approach which is kind of performant. First, put all the points in B into a kD-tree. Now, pick a canonical "first" point in A, and hypothesize matching it to each of the points in B. To check whether it matches a particular point in B, pick a canonical "second" point in A and measure its distance to the "first" point. Then, use a standard kD proximity-bounding query to find all the points in B which are roughly that distance from your hypothesized matched "first" point in B. For each of those, determine the transformation between A and B, and for each of the other points in A, determine whether there's a point in A at roughly the right place (again, using the kD-tree), early-outing with the first unmatched point.
The worst-case performance there can get quite bad with pathological cases (O(n^3 log n), I think) but in general I would expect roughly O(n log n) for well-behaved data with a low threshold. Note that the thresholding is a bit rough-and-ready, and the results can depend on your choice of "first" and "second" points.
This is more of an idea than an answer, but it's too long for a comment. I asked some additional questions in a comment above, but the answers may not be particular relevant, so I'll go ahead and offer some thoughts in the meantime.
As you may know, point matching is its own problem domain, and if you search for 'point matching algorithm', you'll find various articles, papers, and other resources. It seems though that an ad hoc solution might be appropriate here (one that's simpler than more generic algorithms that are available).
I'll assume that the input point set can only be rotated, and not also flipped. If this idea were to work though, it should also work with flipping - you'd just have to run the algorithm separately for each flipped configuration.
In your example image, you've matched a point from set A with a point from set B so that they're coincident. Call this shared point the 'anchor' point. You'd need to do this for every combination of a point from set A and a point from set B until you found a match or exhausted the possibilities. The problem then is to determine if a match can be made given one of these matched point pairs.
It seems that for a given anchor point, a necessary but not sufficient condition for a match is that a point from set A and a point from set B can be found that are approximately the same distance from the anchor point. (What 'approximately' means would depend on the input, and would need to be tuned appropriately given that you're using integers.) This condition is met in your example image in that the center point of each point set is (approximately) the same distance from the anchor point. (Note that there could be multiple pairs of points that meet this condition, in which case you'd have to examine each such pair in turn.)
Once you have such a pair - the center points in your example - you can use some simple trigonometry and linear algebra to rotate set A so that the points in the pair coincide, after which the two point sets are locked together at two points and not just one. In your image that would involve rotating set A about 135 degrees clockwise. Then you check to see if every point in set B has a point in set A with which it's coincident, to within some threshold. If so, you have a match.
In your example, this fails of course, because the rotation is not actually a match. Eventually though, if there's a match, you'll find the anchor point pair for which the test succeeds.
I realize this would be easier to explain with some diagrams, but I'm afraid this written explanation will have to suffice for the moment. I'm not positive this would work - it's just an idea. And maybe a more generic algorithm would be preferable. But, if this did work, it might have the advantage of being fairly straightforward to implement.
[Edit: Perhaps I should add that this is similar to your solution, except for the additional step to allow for only testing a subset of the possible rotations.]
[Edit: I think a further refinement may be possible here. If, after choosing an anchor point, matching is possible via rotation, it should be the case that for every point p in B there's a point in A that's (approximately) the same distance from the anchor point as p is. Again, it's a necessary but not sufficient condition, but it allows you to quickly eliminate cases where a match isn't possible via rotation.]
Below follows a finished solution in python without kD-tree and without early outing candidates. A better way is to do the implementation yourself according to Sneftel but if you need anything quick and with a plot this might be useful.
Plot shows the different steps, starts off with just the template as a collection of connected lines. Then it is translated to a point in B where the distances between A and B points fits the best. Finally it is rotated.
In this example it was important to also match up which of the template positions was matched to which boundingbox position, so its an extra step in the end. There might be some deviations in the code compared to the outline above.
import numpy as np
import random
import math
import matplotlib.pyplot as plt
def to_polar(pos_array):
x = pos_array[:, 0]
y = pos_array[:, 1]
length = np.sqrt(x ** 2 + y ** 2)
t = np.arctan2(y, x)
zip_list = list(zip(length, t))
array_polar = np.array(zip_list)
return array_polar
def to_cartesian(pos):
# first element radius
# second is angle(theta)
# Converting polar to cartesian coordinates
radius = pos[0]
theta = pos[1]
x = radius * math.cos(theta)
y = radius * math.sin(theta)
return x,y
def calculate_distance_points(p1,p2):
return np.sqrt((p1[0]-p2[0])**2+(p1[1]-p2[1])**2)
def find_closest_point_inx(point, neighbour_set):
shortest_dist = None
closest_index = -1
# Find the point in the secondary array that is the closest
for index,curr_neighbour in enumerate(neighbour_set):
distance = calculate_distance_points(point, curr_neighbour)
if shortest_dist is None or distance < shortest_dist:
shortest_dist = distance
closest_index = index
return closest_index
# Find the sum of distances between each point in primary to the closest one in secondary
def calculate_agg_distance_arrs(primary,secondary):
total_distance = 0
for point in primary:
closest_inx = find_closest_point_inx(point, secondary)
dist = calculate_distance_points(point, secondary[closest_inx])
total_distance += dist
return total_distance
# returns a set of <primary_index,neighbour_index>
def pair_neighbours_by_distance(primary_set, neighbour_set, distance_limit):
pairs = {}
for num, point in enumerate(primary_set):
closest_inx = find_closest_point_inx(point, neighbour_set)
if calculate_distance_points(neighbour_set[closest_inx], point) > distance_limit:
closest_inx = None
pairs[num]=closest_inx
return pairs
def rotate_array(array, angle,rot_origin=None):
if rot_origin is not None:
array = np.subtract(array,rot_origin)
# clockwise rotation
theta = np.radians(angle)
c, s = np.cos(theta), np.sin(theta)
R = np.array(((c, -s), (s, c)))
rotated = np.matmul(array, R)
if rot_origin is not None:
rotated = np.add(rotated,rot_origin)
return rotated
# Finds out a point in B_set and a rotation where the points in SetA have the best alignment towards SetB.
def find_stamp_rotation(A_set, B_set):
# Step 1
anchor_point_A = A_set[0]
# Step 2. Convert all points to polar coordinates with anchor as origin
A_anchor_origin = A_set - anchor_point_A
anchor_A_polar = to_polar(A_anchor_origin)
print(anchor_A_polar)
# Step 3 for each point in B
score_tuples = []
for num_anchor, B_anchor_point_try in enumerate(B_set):
# Step 3.1
B_origin_rel_point = B_set-B_anchor_point_try
B_polar_rp_origin = to_polar(B_origin_rel_point)
# Step 3.3 select arbitrary point q from Ap
point_Aq = anchor_A_polar[1]
# Step 3.4 test each rotation, where pointAq is rotated to each B-point (except the B anchor point)
for try_rot_point_B in [B_rot_point for num_rot, B_rot_point in enumerate(B_polar_rp_origin) if num_rot != num_anchor]:
# positive rotation is clockwise
# Step 4.1 Rotate Ap by the angle between q and n
angle_to_try = try_rot_point_B[1]-point_Aq[1]
rot_try_arr = np.copy(anchor_A_polar)
rot_try_arr[:,1]+=angle_to_try
cart_rot_try_arr = [to_cartesian(e) for e in rot_try_arr]
cart_B_rp_origin = [to_cartesian(e) for e in B_polar_rp_origin]
distance_score = calculate_agg_distance_arrs(cart_rot_try_arr, cart_B_rp_origin)
score_tuples.append((B_anchor_point_try,angle_to_try,distance_score))
# Step 4.3
lowest=None
for b_point,angle,distance in score_tuples:
print("point:{} angle(rad):{} distance(sum):{}".format(b_point,360*(angle/(2*math.pi)),distance))
if lowest is None or distance < lowest[2]:
lowest = b_point, 360*angle/(2*math.pi), distance
return lowest
def test_example():
ax = plt.subplot()
ax.grid(True)
plt.title('Fit Template to BBoxes by translation and rotation')
plt.xlim(-20, 20)
plt.ylim(-20, 20)
ax.set_xticks(range(-20,20), minor=True)
ax.set_yticks(range(-20,20), minor=True)
template = np.array([[-10,-10],[-10,10],[0,0],[10,-10],[10,10], [0,20]])
# Test Bboxes are Rotated 40 degree, translated 2,2
rotated = rotate_array(template,40)
rotated = np.subtract(rotated,[2,2])
# Adds some extra bounding boxes as noise
for i in range(8):
rotated = np.append(rotated,[[random.randrange(-20,20), random.randrange(-20,20)]],axis=0)
# Scramble entries in array and return the position change.
rnd_rotated = rotated.copy()
np.random.shuffle(rnd_rotated)
element_positions = []
# After shuffling, looks at which index the "A"-marks has ended up at. For later comparison to see that the algo found the correct answer.
# This is to represent the actual case, where I will get a bunch of unordered bboxes.
rnd_map = {}
indexes_translation = [num2 for num,point in enumerate(rnd_rotated) for num2,point2 in enumerate(rotated) if point[0]==point2[0] and point[1]==point2[1]]
for num,inx in enumerate(indexes_translation):
rnd_map[num]=inx
# algo part 1/3
b_point,angle,_ = find_stamp_rotation(template,rnd_rotated)
# Plot for visualization
legend_list = np.empty((0,2))
leg_template = plt.plot(template[:,0],template[:,1],c='r')
legend_list = np.append(legend_list,[[leg_template[0],'1. template-pattern']],axis=0)
leg_bboxes = plt.scatter(rnd_rotated[:,0],rnd_rotated[:,1],c='b',label="scatter")
legend_list = np.append(legend_list,[[leg_bboxes,'2. bounding boxes']],axis=0)
leg_anchor = plt.scatter(b_point[0],b_point[1],c='y')
legend_list = np.append(legend_list,[[leg_anchor,'3. Discovered bbox anchor point']],axis=0)
# algo part 2/3
# Superimpose A onto B by A[0] to b_point
offset = b_point - template[0]
super_imposed_A = template + offset
# Plot superimposed, but not yet rotated
leg_s_imposed = plt.plot(super_imposed_A[:,0],super_imposed_A[:,1],c='k')
#plt.legend(rubberduckz, "superimposed template on anchor")
legend_list = np.append(legend_list,[[leg_s_imposed[0],'4. Templ superimposed on Bbox']],axis=0)
print("Superimposed A on B by A[0] to {}".format(b_point))
print(super_imposed_A)
# Rotate, now the template should match pattern of bboxes
# algo part 3/4
super_imposed_rotated_A = rotate_array(super_imposed_A,-angle,rot_origin=super_imposed_A[0])
# Show the beautiful match in a last plot
leg_s_imp_rot = plt.plot(super_imposed_rotated_A[:,0],super_imposed_rotated_A[:,1],c='g')
legend_list = np.append(legend_list,[[leg_s_imp_rot[0],'5. final fit']],axis=0)
plt.legend(legend_list[:,0], legend_list[:,1],loc="upper left")
plt.show()
# algo part 4/4
pairs = pair_neighbours_by_distance(super_imposed_rotated_A, rnd_rotated, 10)
print(pairs)
for inx in range(len(pairs)):
bbox_num = pairs[inx]
print("template id:{}".format(inx))
print("bbox#id:{}".format(bbox_num))
#print("original_bbox:{}".format(rnd_map[bbox_num]))
if __name__ == "__main__":
test_example()
Result on actual image with bounding boxes. Here it can be seen that the scaling is incorrect which makes the template a bit off but it will still be able to pair up and thats the desired end-result in my case.
I am looking for a tool similar to graphviz that can render graphs, but that will allow me to constrain just the x coordinate of each node. Then, the tool will automatically choose y coordinates to make the graph look neat.
Basically, I want to make a timeline.
Language / platform / rendering medium are not very important.
If you want a neat-looking graph a force-directed algorithm is going to be your best bet. One of the best ones is SFDP (developed by AT&T, included in graphviz) though I can't seem to find pseudocode or an easy implementation. I don't think there are any algorithms this specialized. Thankfully, it's easy to code your own. I'll present some pseudocode mostly lifted form Wikipedia, but with suitably one-dimensional modifications. I'll assume you have n vertices and the vector of x-positions is x, subscripted by x.i.
set all vertex velocities to (0,0)
set all vertex positions to (x.i, random)
while (KE > epsilon)
KE = 0
for each vertex v
force = (0,0)
for each vertex u != v
force = force + (0, coulomb(u, v).y)
if u is incident to v
force = force + (0, hooke(u, v).y)
v.velocity = (v.velocity + timestep * force) * damping
v.position = v.position + timestep * v.velocity
KE = KE + |v.velocity| ^ 2
here the .y denotes getting the y-component of the force. This ensures that the x-components of the positions of the vertices never change from what you set them to be. The epsilon parameter is to be set by you, and should be something small compared to what you expect KE (the kinetic energy) to be. Also, |v| denotes the magnitude of the vector v (all computations are of 2-vectors in the above, except the KE). Note I set the mass of all the nodes to be 1, but you can change that if you want.
The Hooke and Coulomb functions calculate the respective forces between nodes; the first is linear in distance between vertices, the second is quadratic, so there is a guaranteed equilibrium. These functions look something like
def hooke(u, v)
return -k * |u.position - v.position|
def coulomb(u, v)
return C * |u.position - v.position|
where again most computations are in vector form. C and k have real values but experiment to get the graph you want. This isn't usually necessary because the scaling factors will, in two dimensions, pretty much expand or contract the whole graph, but here the x-distances are set so to get a good-looking graph you will have to change the values a bit.
What is the best way to control color grading having a start and an end color? Using RGB or HSV ?
E.g
.
step = 1/nrColors;
currStep = 0;
for i = 1:nrColors
p = sigmoid(currStep);
R = start.R * p + end.R * (1-p)
G = ...
B = ...
currStep = currStep + step ;
end
or
H = start.H * p + end.R * (1-p)
S = ...
V = ...
With neither of the above, I could achieve a good transtion (either the colors are too tight to each other, or the colors are not smooth from humans point of view).
Where could I find a good reference on the subject ?
Best,
Theoretically, you will get the best match to human perception if you interpolate in the "Lab" color space:
http://en.wikipedia.org/wiki/CIELAB
Also, a sigmoid interpolation may be appropriate if you are animating a color transition, but if you are displaying a sequence of swatches that smoothly interpolate from one color to another (which is what it sounds like given that you talk about them being too tight to each other), then you might just want to interpolate linearly. Otherwise the start and end colors will be very similar, and the center ones more spaced out.
I have a hunch this has been done before but I am a total layman at this and don't know how to begin to ask the right question. So I will describe what I am trying to do...
I have an unknown ARGB color. I only know its absolute RGB value as displayed over two known opaque background colors, for example black 0x000000 and white 0xFFFFFF. So, to continue the example, if I know that the ARGB color is RGB 0x000080 equivalent when displayed over 0x000000 and I know that the same ARGB color is RGB 0x7F7FFF equivalent when displayed over 0xFFFFFF, is there a way to compute what the original ARGB color is?
Or is this even possible???
So, you know that putting (a,r,g,b) over (r1,g1,b1) gives you (R1,G1,B1) and that putting it over (r2,g2,b2) gives you (R2,G2,B2). In other words -- incidentally I'm going to work here in units where a ranges from 0 to 1 -- you know (1-a)r1+ar=R1, (1-a)r2+ar=R2, etc. Take those two and subtract: you get (1-a)(r1-r2)=R1-R2 and hence a=1-(R1-R2)/(r1-r2). Once you know a, you can work everything else out.
You should actually compute the values of a you get from doing that calculation on all three of {R,G,B} and average them or something, to reduce the effects of roundoff error. In fact I'd recommend that you take a = 1 - [(R1-R2)sign(r1-r2) + (G1-G2)sign(g1-g2) + (B1-B2)sign(b1-b2)] / (|r1-r2|+|g1-g2|+|b1-b2), which amounts to weighting the more reliable colours more highly.
Now you have, e.g., r = (R1-(1-a)r1)/a = (R2-(1-a)r2)/a. These two would be equal if you had infinite-precision values for a,r,g,b, but of course in practice they may differ slightly. Average them: r = [(R1+R2)-(1-a)(r1+r2)]/2a.
If your value of a happens to be very small then you'll get only rather unreliable information about r,g,b. (In the limit where a=0 you'll get no information at all, and there's obviously nothing you can do about that.) It's possible that you may get numbers outside the range 0..255, in which case I don't think you can do better than just clipping.
Here's how it works out for your particular example. (r1,g1,b1)=(0,0,0); (r2,g2,b2)=(255,255,255); (R1,G1,B1)=(0,0,128); (R2,G2,B2)=(127,127,255). So a = 1 - [127+127+127]/[255+255+255] = 128/255, which happens to be one of the 256 actually-possible values of a. (If it weren't, we should probably round it at this stage.)
Now r = (127-255*127/255)*255/256 = 0; likewise g = 0; and b = (383-255*127/255)*255/256 = 255.
So our ARGB colour was 80,00,00,FF.
Choosing black and white as the background colors is the best choice, both for ease of calculation and accuracy of result. With lots of abuse of notation....
a(RGB) + (1-a)0xFFFFFF = 0x7F7FFF
a(RGB) + (1-a)0x000000 = 0x000080
Subtracting the second from the first...
(1-a)0xFFFFFF = 0x7F7FFF-0x000080 = 0x7F7F7F
So
(1-a) = 0x7F/0xFF
a = (0xFF-0x7F)/0xFF = 0x80/0xFF
A = 0x80
and RGB = (a(RGB))/a = 0x000080/a = 0x0000FF
You can do something very similar with other choices of background color. The smaller a is and the closer the two background colors are the less accurately you will be able to determine the RGBA value. Consider the extreme cases where A=0 or where the two background colors are the same.
I'm trying to implement this extenstion of the Karplus-Strong plucked string algorithm, but I don't understand the notation there used. Maybe it will take years of study, but maybe it won't - maybe you can tell me.
I think the equations below are in the frequency domain or something. Just starting with the first equation, Hp(z), the pick direction lowpass filter. For one direction you use p = 0, for the other, perhaps 0.9. This boils down to to 1 in the first case, or 0.1 / (1 - 0.9 z-1) in the second.
alt text http://www.dsprelated.com/josimages/pasp/img902.png
Now, I feel like this might mean, in coding terms, something towards:
H_p(float* input, int time) {
if (downpick) {
return input[time];
} else {
return some_function_of(input[t], input[t-1]);
}
}
Can someone give me a hint? Or is this futile and I really need all the DSP background to implement this? I was a mathematician once...but this ain't my domain.
So the z-1 just means a one-unit delay.
Let's take Hp = (1-p)/(1-pz-1).
If we follow the convention of "x" for input and "y" for output, the transfer function H = y/x (=output/input)
so we get y/x = (1-p)/(1-pz-1)
or (1-p)x = (1-pz-1)y
(1-p)x[n] = y[n] - py[n-1]
or: y[n] = py[n-1] + (1-p)x[n]
In C code this can be implemented
y += (1-p)*(x-y);
without any additional state beyond using the output "y" as a state variable itself. Or you can go for the more literal approach:
y_delayed_1 = y;
y = p*y_delayed_1 + (1-p)*x;
As far as the other equations go, they're all typical equations except for that second equation which looks like maybe it's a way of selecting either HΒ = 1-z-1 OR 1-z-2. (what's N?)
The filters are kind of vague and they'll be tougher for you to deal with unless you can find some prepackaged filters. In general they're of the form
H = H0*(1+az-1+bz-2+cz-3...)/(1+rz-1+sz-2+tz-3...)
and all you do is write down H = y/x, cross multiply to get
H0 * (1+az-1+bz-2+cz-3...) * x = (1+rz-1+sz-2+tz-3...) * y
and then isolate "y" by itself, making the output "y" a linear function of various delays of itself and of the input.
But designing filters (picking the a,b,c,etc.) is tougher than implementing them, for the most part.