Pair two sets such that the distance between elements is minimized - combinatorics

I have two sets S_1 and S_2. Given these two sets, I need to pair each element from S_1 with an element from S_2.
The elements are not reusable, so if S_1[A] is paired with S_2[D], then I cannot also pair S_1[B] with S_2[D].
The goal is to produce a pairing using all elements such that the distance of the pairing is minimized.
The distance of the pairing is computed as the sum of the distance between each pair.
Produce result with lowest total paired points value
Are there any known algorithms for solving this type of problem efficiently?
Part of the difficulty is that taking a greedy approach doesn't work. If S_1 = [A, B, C] and S_2 = [D, E, F], and distance(A, D) = 0.1, distance(A, E) = 0.3, distance(A, F) = 0.4, you can't naively match A to D just because it has the lowest distance for this set. Suppose that distance(B, D) = 0.1, distance(B, E) = 0.8, and distance(B, F) = 0.9. If you naively choose to match (A, D) in the first iteration, then you actually make the overall distance higher because this forces you to match either (B, E) or (B, D). It would be a better choice to match (A, E) and then allow (B, D) to match. This means you can't iterate over S_1 and greedily assign matches based on the lowest distance between each element of S_1 and the remaining elements of S_2.
This seems similar to the assignment problem, which I could solve using something like the Hungarian Algorithm (https://en.wikipedia.org/wiki/Hungarian_algorithm), but I believe that algorithm allows reusing elements, which won't work for my case.

Related

Python code yielding different result for same numerical value, depending on inclusion of precision point

I defined a function which returns a third order polynomial function for either a value, a list or a np.array:
def two_d_third_order(x, a, b, c, d):
return a + np.multiply(b, x) + np.multiply(c, np.multiply(x, x)) + np.multiply(d, np.multiply(x, np.multiply(x, x)))
The issue I noticed is, however, when I use "two_d_third_order" on the following two inputs:
1500
1500.0
With (a, b, c, d) = (1.20740028e+00, -2.93682465e-03, 2.29938078e-06, -5.09134552e-10), I get two different results:
2.4441
0.2574
, respectively. I don't know how this is possible, and any help would be appreciated.
I tried several inputs, and somehow the inclusion of a floating point on certain values (despite representing the same numerical value) changes the end result.

Python uses implicit data type conversions. When you use only integers (like 1500), there is a loss of precision in all subsequent operations. Whereas when you pass it a float or double (like 1500.0), subsequent operations are performed with the associated datatype, i.e in this case with higher precision.
This is not a "bug" so to speak, but generally how Python operates without the explicit declaration of data types. Languages like C and C++ require explicit data type declarations and explicit data type casting to ensure operations are performed in the prescribed precision formats. Can be a boon or a bane depending on usage.

1500 * 2_250_000 is 3_375_000_000 overflowing the range on int32:
print(type(np.multiply(1500, 2250000)))
print(np.multiply(1500, 2250000))
giving:
<class 'numpy.int32'>
-919967296
Where as floats use a much larger container.
print(type(np.multiply(1500.0, 2250000.0)))
print(np.multiply(1500.0, 2250000.0))
giving:
<class 'numpy.float64'>
3375000000.0
Try casting your input to a larger int.
(a, b, c, d) = (1.20740028e+00, -2.93682465e-03, 2.29938078e-06, -5.09134552e-10)
x1 = np.int64(1500)
x2 = 1500.0
print(two_d_third_order(x1, a, b, c, d) == two_d_third_order(x2, a, b, c, d))

How to find the L1-Norm/Manhattan distance between two vectors in Python without libraries

I have two vectors with equal dimensions and need to find the distance between them
I have tried various approaches:
sum([a-b for a, b in zip(u, v)])
c= sum([a-b for a, b in zip(u, v)]
#If x is negative, multiply by negative one to convert x to a positive
if c<=0:
return c*-1
#No changes are made to x if it is positive
else:
return c
I am yet to have success!

You want to use the abs() function, which is available in standard python.
So if you have
a = [1,2,3,4,5,.4]
b = [4,3,4,5,-2,.8]
Than you can get the distance with
sum([abs(i-j) for i,j in zip(a,b)])
We can use the sklearn implementation to check indeed this is the correct answer.
from sklearn.metrics.pairwise import manhattan_distances
manhattan_distances([a], [b])

Prolog ways to compare variables

I was trying to implement some graph algorithms in Prolog. I came up with an idea to use unification to build a tree from the graph structure:
The graph would be defined as follows:
A list of Vertex-Variable pairs where Vertex is a constant representing the vertex and Variable is a corresponding variable, that would be used as a "reference" to the vertex. e.g.:
[a-A, b-B, c-C, d-D]
A list of VertexVar-NeighboursList pairs, where the VertexVar and the individual neighbours in the NeighboursList are the "reference variables". e.g.:
[A-[B, C, D], B-[A, C], C-[A, B], D-[A]] meaning b, c, d are neighbours of a etc.
Then before some graph algorithm (like searching for components, or simple DFS/BFS etc.) that could use some kind of tree built from the original graph, one could use some predicate like unify_neighbours that unifies the VertexVar-NeighbourList pairs as VertexVar = NeighboursList. After that, the vertex variables may be interpreted as lists of its neighbours, where each neighbour is again a list of its neighbours.
So this would result in a good performance when traversing the graph, as there is no need in linear search for some vertex and its neighbours for every vertex in the graph.
But my problem is: How to compare those vertex variables? (To check if they're the same.) I tried to use A == B, but there are some conflicts. For the example above, (with the unify_neighbours predicate) Prolog interprets the graph internally as:
[a-[S_1, S_2, S_3], b-S_1, c-S_2, d-S_3]
where:
S_1 = [[S_1, S_2, S_3], S_2]
S_2 = [[S_1, S_2, S_3], S_1]
S_3 = [[S_1, S_2, S_3]]
The problem is with S_1 and S_2 (aka b and c) as X = [something, Y], Y = [something, X], X == Y is true. The same problem would be with vertices, that share the same neighbours. e.g. U-[A, B] and V-[A, B].
So my question is: Is there any other way to compare variables, that could help me with this? Something that compares "the variables themselves", not the content, like comparing addresses in procedural programming languages? Or would that be too procedural and break the declarative idea of Prolog?
Example
graph_component(Vertices, Neighbours, C) :-
% Vertices and Neighbours as explained above.
% C is some component found in the graph.
vertices_refs(Vertices, Refs),
% Refs are only the variables from the pairs.
unify_neighbours(Neighbours), % As explained above.
rec_(Vertices, Refs, [], C).
rec_(Vertices, Refs, Found, RFound) :-
% Vertices as before.
% Refs is a stack of the vertex variables to search.
% Found are the vertices found so far.
% RFound is the resulting component found.
[Ref|RRest] = Refs,
vertices_pair(Vertices, Vertex-Ref),
% Vertex is the corresponding Vertex for the Ref variable
not(member(Vertex, Found)),
% Go deep:
rec_(Vertices, Ref, [Vertex|Found], DFound),
list_revpush_result([Vertex|Found], DFound, Found1),
% Go wide:
rec_(Vertices, RRest, Found1, RFound).
rec_(Vertices, Refs, Found, []) :-
% End of reccursion.
[Ref|_] = Refs,
vertices_pair(Vertices, Vertex-Ref),
member(Vertex, Found).
This example doesn't really work, but it's the idea. (Also, checking whether the vertices were found is done linearly, so the performance is still not good, but it's just for demonstration.) Now the predicate, that finds the corresponding vertex for the variable is implemented as:
vertices_pair([Vertex-Ref|_], Vertex-Ref).
vertices_pair([_-OtherRef|Rest], Vertex-Ref) :-
Ref \== OtherRef,
vertices_pair(Rest, Vertex-Ref).
where the \== operator is not really what I want and it creates those conflicts.

It is an intrinsic feature of Prolog that, once you have bound a variable to a term, it becomes indistinguishable from the term itself. In other words, if you bind two variables to the same term, you have two identical things, and there is no way to tell them apart.
Applied to your example: once you have unified every vertex-variable with the corresponding neighbours-list, all the variables are gone: you are left simply with a nested (and most likely circular) data structure, consisting of a list of lists of lists...
But as you suggest, the nested structure is an attractive idea because it gives you direct access to adjacent nodes. And although Prolog system vary somewhat in how well they support circular data structures, this need not stop you from exploiting this idea.
The only problem with your design is that a node is identified purely by the (potentially deeply nested and circular) data structure that describes the sub-graph that is reachable from it. This has the consequence that
two nodes that have the same descendants are indistinguishable
it can be very expensive to check whether two "similar looking" sub-graphs are identical or not
A simple way around that is to include a unique node identifier (such as a name or number) in your data structure. To use your example (slightly modified to make it more interesting):
make_graph(Graph) :-
Graph = [A,B,C,D],
A = node(a, [C,D]),
B = node(b, [A,C]),
C = node(c, [A,B]),
D = node(d, [A]).
You can then use that identifier to check for matching nodes, e.g. in a depth-first traversal:
dfs_visit_nodes([], Seen, Seen).
dfs_visit_nodes([node(Id,Children)|Nodes], Seen1, Seen) :-
( member(Id, Seen1) ->
Seen2 = Seen1
;
writeln(visiting(Id)),
dfs_visit_nodes(Children, [Id|Seen1], Seen2)
),
dfs_visit_nodes(Nodes, Seen2, Seen).
Sample run:
?- make_graph(G), dfs_visit_nodes(G, [], Seen).
visiting(a)
visiting(c)
visiting(b)
visiting(d)
G = [...]
Seen = [d, b, c, a]
Yes (0.00s cpu)

Thanks, #jschimpf, for the answer. It clarified a lot of things for me. I just got back to some graph problems with Prolog and thought I'd give this recursive data structure another try and came up with the following predicates to construct this data structure from a list of edges:
The "manual" creation of the data structure, as proposed by #jschimpf:
my_graph(Nodes) :-
Vars = [A, B, C, D, E],
Nodes = [
node(a, [edgeTo(1, B), edgeTo(5, D)]),
node(b, [edgeTo(1, A), edgeTo(4, E), edgeTo(2, C)]),
node(c, [edgeTo(2, B), edgeTo(6, F)]),
node(d, [edgeTo(5, A), edgeTo(3, E)]),
node(e, [edgeTo(3, D), edgeTo(4, B), edgeTo(1, F)]),
node(e, [edgeTo(1, E), edgeTo(6, C)])
],
Vars = Nodes.
Where edgeTo(Weight, VertexVar) represents an edge to some vertex with a weight assosiated with it. The weight is just to show that this can be customized for any additional information. node(Vertex, [edgeTo(Weight, VertexVar), ...]) represents a vertex with its neighbours.
A more "user-friendly" input format:
[edge(Weight, FromVertex, ToVertex), ...]
With optional list of vertices:
[Vertex, ...]
For the example above:
[edge(1, a, b), edge(5, a, d), edge(2, b, c), edge(4, b, e), edge(6, c, f), edge(3, d, e), edge(1, e, f)]
This list can be converted to the recursive data structure with the following predicates:
% make_directed_graph(+Edges, -Nodes)
make_directed_graph(Edges, Nodes) :-
vertices(Edges, Vertices),
vars(Vertices, Vars),
pairs(Vertices, Vars, Pairs),
nodes(Pairs, Edges, Nodes),
Vars = Nodes.
% make_graph(+Edges, -Nodes)
make_graph(Edges, Nodes) :-
vertices(Edges, Vertices),
vars(Vertices, Vars),
pairs(Vertices, Vars, Pairs),
directed(Edges, DiretedEdges),
nodes(Pairs, DiretedEdges, Nodes),
Vars = Nodes.
% make_graph(+Edges, -Nodes)
make_graph(Edges, Nodes) :-
vertices(Edges, Vertices),
vars(Vertices, Vars),
pairs(Vertices, Vars, Pairs),
directed(Edges, DiretedEdges),
nodes(Pairs, DiretedEdges, Nodes),
Vars = Nodes.
% make_directed_graph(+Vertices, +Edges, -Nodes)
make_directed_graph(Vertices, Edges, Nodes) :-
vars(Vertices, Vars),
pairs(Vertices, Vars, Pairs),
nodes(Pairs, Edges, Nodes),
Vars = Nodes.
The binary versions of these predicates assume, that every vertex can be obtained from the list of edges only - There are no "edge-less" vertices in the graph. The ternary versions take an additional list of vertices for exactly these cases.
make_directed_graph assumes the input edges to be directed, make_graph assumes them to be undirected, so it creates additional directed edges in the opposite direction:
% directed(+UndirectedEdges, -DiretedEdges)
directed([], []).
directed([edge(W, A, B)|UndirectedRest], [edge(W, A, B), edge(W, B, A)|DirectedRest]) :-
directed(UndirectedRest, DirectedRest).
To get all the vertices from the list of edges:
% vertices(+Edges, -Vertices)
vertices([], []).
vertices([edge(_, A, B)|EdgesRest], [A, B|VerticesRest]) :-
vertices(EdgesRest, VerticesRest),
\+ member(A, VerticesRest),
\+ member(B, VerticesRest).
vertices([edge(_, A, B)|EdgesRest], [A|VerticesRest]) :-
vertices(EdgesRest, VerticesRest),
\+ member(A, VerticesRest),
member(B, VerticesRest).
vertices([edge(_, A, B)|EdgesRest], [B|VerticesRest]) :-
vertices(EdgesRest, VerticesRest),
member(A, VerticesRest),
\+ member(B, VerticesRest).
vertices([edge(_, A, B)|EdgesRest], VerticesRest) :-
vertices(EdgesRest, VerticesRest),
member(A, VerticesRest),
member(B, VerticesRest).
To construct uninitialized variables for every vertex:
% vars(+List, -Vars)
vars([], []).
vars([_|ListRest], [_|VarsRest]) :-
vars(ListRest, VarsRest).
To pair up verticies and vertex variables:
% pairs(+ListA, +ListB, -Pairs)
pairs([], [], []).
pairs([AFirst|ARest], [BFirst|BRest], [AFirst-BFirst|PairsRest]) :-
pairs(ARest, BRest, PairsRest).
To construct the recursive nodes:
% nodes(+Pairs, +Edges, -Nodes)
nodes(Pairs, [], Nodes) :-
init_nodes(Pairs, Nodes).
nodes(Pairs, [EdgesFirst|EdgesRest], Nodes) :-
nodes(Pairs, EdgesRest, Nodes0),
insert_edge(Pairs, EdgesFirst, Nodes0, Nodes).
First, a list of empty nodes for every vertex is initialized:
% init_nodes(+Pairs, -EmptyNodes)
init_nodes([], []).
init_nodes([Vertex-_|PairsRest], [node(Vertex, [])|NodesRest]) :-
init_nodes(PairsRest, NodesRest).
Then the edges are inserted one by one:
% insert_edge(+Pairs, +Edge, +Nodes, -ResultingNodes)
insert_edge(Pairs, edge(W, A, B), [], [node(A, [edgeTo(W, BVar)])]) :-
vertex_var(Pairs, B, BVar).
insert_edge(Pairs, edge(W, A, B), [node(A, EdgesTo)|NodesRest], [node(A, [edgeTo(W, BVar)|EdgesTo])|NodesRest]) :-
vertex_var(Pairs, B, BVar).
insert_edge(Pairs, edge(W, A, B), [node(X, EdgesTo)|NodesRest], [node(X, EdgesTo)|ResultingNodes]) :-
A \= X,
insert_edge(Pairs, edge(W, A, B), NodesRest, ResultingNodes).
To get a vertex variable for a given vertex: (This actually works in both directions.)
% vertex_var(+Pairs, +Vertex, -Var)
vertex_var(Pairs, Vertex, Var) :-
member(Vertex-Var, Pairs).
```Prolog
This, of course, brings additional time overhead, but you can do this once and then just copy this data structure every time you need to perform some graph algorithm on it and access neighbours in constant time.
You can also add additional information to the `node` predicate. For example:
```Prolog
node(Vertex, Neighbours, OrderingVar)
Where the uninitialized variable OrderingVar can be "assigned" (initialized) in constant time with information about the vertex' position in a partial ordering of the graph, for example. So this may be used as output. (As sometimes denoted by +- in Prolog comments - an uninitialized variable as a part of an input term, that is yet to be initialized by the used predicate and provides output.)

Matrix-vector multiplication for only one dimension in a tensor

Is it possible to multiply only one (last) dimension in a tensor alone with other vectors?
For example, assume a tensor T=[100, 20, 400] and a matrix M =[400, 400].
Is it possible to make the operation h_{transpose}*M*h, where h is the last dimension in the tensor T? In other words, is it possible to make use of (possibly pytorch) built-in functions to get the resulting tensor of size [100, 20, 1]?

I think the easiest (certainly the shortest) solution is with einsum.
import torch
T = torch.randn(100, 20, 400)
M = torch.randn(400, 400)
res = torch.einsum('abc,cd,abd->ab', (T, M, T)).unsqueeze(-1)
It basically says "for all (a, b, c, d) in bounds, multiply T[a, b, c] with M[c, d] and T[a, b, d] and accumulate it in res[a, b]".
Since einsum is implemented in terms of basic building blocks like mm, transpose etc, this could certainly be unrolled into a more "classical" solution, but right now my brain fails me at that.

Edit distance, with a twist

I'm trying to solve something using dynamic programming, but I'm having some trouble. When I work on dynamic programming, I usually determine a recursive algorithm then go from there to my dynamic solution. This time I'm having trouble
The Problem
Say you have two strings: m and n, such that n.length is greater than m.length, and n does not contain the character '#'. You want the string that turns m into the same length as string n in minimum cost.
Cost is defined as SUM(Penalty(m[i],n[i])), where i is in an index of the strings char array.
Penalty is defined as such
private static int penalty(char x,char y) {
if (x==y) { return 0;}
else if (y=='#') { return 4;}
else { return 2;}
}
The only way I can think of is as follows:
[0] If m and n are the same length, return m
[1] Compute cost of inserting a # at any index of m
[2] determine the string that has the minimum of such cost. Let that string be m'
[3] Run the algorithm on m' and n again.
I don't think this is even the optimal recursive algorithm, leading me to believe that I'm not on the right track for a dynamic algorithm.
I've read up on using a m.length x n.length matrix for normal edit distance, but I don't see how I could easily transform that to fit my algorithm.
Thoughts on my recursive algorithm and the steps I need to take to reach a dynamic solution?

Taking your definitions (python):
def penalty(x, y):
if x == y:
return 0
if y == '#':
return 4
return 2
def cost(n, m):
return sum(penalty(a, b) for a, b in zip(n, m))
Then you can define the distance reassigning to m the lowest cost for each # to be included.
def distance(n, m):
for _ in range(len(n) - len(m)):
m = min((m[:i]+'#'+m[i:] for i in range(len(m)+1)), key=lambda s: cost(n, s))
return m
>>> distance('hello world', 'heloworld')
'he#lo#world'

The only way that I can see the optimality principle to work here is if you solve the problem over growing lengths of n. So the dynamic programming solution would look like this:
For each contiguous substring of length m.length()+1, solve the problem, yielding a list of proposals for the new m.
Select the proposal with the minimum distance to the corresponding substring as the new m, and repeat the process.
You won't need to store anything other than the currently optimal solution in this algorithm, certainly not a distance matrix. It looks to me like you were pretty close to this solution as well, you only missed the 'shrink n to get a subproblem'-part.

Resources