Related
I have a list with 10 numerical values. I want to return all possible combination of this list such that each element can take value +/- element value.
The approach I had in mind was to take a binary variable which takes in value from 0 to 1023. 1 in this variable corresponds to positive d[i] and 0 to negative d[i].
e.g. for bin(8) = 0000001000 implies that d7 will take value -d7 and rest will be positive. Repeat this for all 0 to 1023 to get all combinations.
For example, if D = [d1,d2,...d10], we will have 1024 (2^10) combinations such that:
D1 = [-d1,d2,d3,....d10]
D2 = [-d1,-d2,d3,....d10]
D3 = [d1,-d2,d3,....d10] ...
D1024 = [-d1,-d1,-d3,....-d10]
Thank You!
you can just use the builtin itertools.product to make all combinations of positive and negative values.
from itertools import product
inputs = list(range(10)) # [1,2,3,4,5,6,7,8,9]
outputs = []
choices = [(x,-x) for x in inputs]
for item in product(*choices):
outputs.append(item)
print(outputs[:3])
print(len(outputs))
# [(0, 1, 2, 3, 4, 5, 6, 7, 8, 9), (0, 1, 2, 3, 4, 5, 6, 7, 8, -9), (0, 1, 2, 3, 4, 5, 6, 7, -8, 9)]
# 1024
in a compressed form:
outputs = [item for item in product(*[(x,-x) for x in inputs])]
I have the following Python dict:
[(2, [3, 4, 5]), (3, [1, 0, 0, 0, 1]), (4, [-1]), (10, [1, 2, 3])]
Now I want to sort them on the basis of sum of values of the values of dictionary, so for the first key the sum of values is 3+4+5=12.
I have written the following code that does the job:
def myComparator(a,b):
print "Values(a,b): ",(a,b)
sum_a=sum(a[1])
sum_b=sum(b[1])
print sum_a,sum_b
print "Comparision Returns:",cmp(sum_a,sum_b)
return cmp(sum_a,sum_b)
items.sort(myComparator)
print items
This is what the output that I get after running above:
Values(a,b): ((3, [1, 0, 0, 0, 1]), (2, [3, 4, 5]))
2 12
Comparision Returns: -1
Values(a,b): ((4, [-1]), (3, [1, 0, 0, 0, 1]))
-1 2
Comparision Returns: -1
Values(a,b): ((10, [1, 2, 3]), (4, [-1]))
6 -1
Comparision Returns: 1
Values(a,b): ((10, [1, 2, 3]), (3, [1, 0, 0, 0, 1]))
6 2
Comparision Returns: 1
Values(a,b): ((10, [1, 2, 3]), (2, [3, 4, 5]))
6 12
Comparision Returns: -1
[(4, [-1]), (3, [1, 0, 0, 0, 1]), (10, [1, 2, 3]), (2, [3, 4, 5])]
Now I am unable to understand as to how the comparator is working, which two values are being passed and how many such comparisons would happen? Is it creating a sorted list of keys internally where it keeps track of each comparison made? Also the behavior seems to be very random. I am confused, any help would be appreciated.
The number and which comparisons are done is not documented and in fact, it can freely change from different implementations. The only guarantee is that if the comparison function makes sense the method will sort the list.
CPython uses the Timsort algorithm to sort lists, so what you see is the order in which that algorithm is performing the comparisons (if I'm not mistaken for very short lists Timsort just uses insertion sort)
Python is not keeping track of "keys". It just calls your comparison function every time a comparison is made. So your function can be called many more than len(items) times.
If you want to use keys you should use the key argument. In fact you could do:
items.sort(key=lambda x: sum(x[1]))
This will create the keys and then sort using the usual comparison operator on the keys. This is guaranteed to call the function passed by key only len(items) times.
Given that your list is:
[a,b,c,d]
The sequence of comparisons you are seeing is:
b < a # -1 true --> [b, a, c, d]
c < b # -1 true --> [c, b, a, d]
d < c # 1 false
d < b # 1 false
d < a # -1 true --> [c, b, d, a]
how the comparator is working
This is well documented:
Compare the two objects x and y and return an integer according to the outcome. The return value is negative if x < y, zero if x == y and strictly positive if x > y.
Instead of calling the cmp function you could have written:
sum_a=sum(a[1])
sum_b=sum(b[1])
if sum_a < sum_b:
return -1
elif sum_a == sum_b:
return 0
else:
return 1
which two values are being passed
From your print statements you can see the two values that are passed. Let's look at the first iteration:
((3, [1, 0, 0, 0, 1]), (2, [3, 4, 5]))
What you are printing here is a tuple (a, b), so the actual values passed into your comparison functions are
a = (3, [1, 0, 0, 0, 1])
b = (2, [3, 4, 5]))
By means of your function, you then compare the sum of the two lists in each tuple, which you denote sum_a and sum_b in your code.
and how many such comparisons would happen?
I guess what you are really asking: How does the sort work, by just calling a single function?
The short answer is: it uses the Timsort algorithm, and it calls the comparison function O(n * log n) times (note that the actual number of calls is c * n * log n, where c > 0).
To understand what is happening, picture yourself sorting a list of values, say v = [4,2,6,3]. If you go about this systematically, you might do this:
start at the first value, at index i = 0
compare v[i] with v[i+1]
If v[i+1] < v[i], swap them
increase i, repeat from 2 until i == len(v) - 2
start at 1 until no further swaps occurred
So you get, i =
0: 2 < 4 => [2, 4, 6, 3] (swap)
1: 6 < 4 => [2, 4, 6, 3] (no swap)
2: 3 < 6 => [2, 4, 3, 6] (swap)
Start again:
0: 4 < 2 => [2, 4, 3, 6] (no swap)
1: 3 < 4 => [2, 3, 4, 6] (swap)
2: 6 < 4 => [2, 3, 4, 6] (no swap)
Start again - there will be no further swaps, so stop. Your list is sorted. In this example we have run through the list 3 times, and there were 3 * 3 = 9 comparisons.
Obviously this is not very efficient -- the sort() method only calls your comparator function 5 times. The reason is that it employs a more efficient sort algorithm than the simple one explained above.
Also the behavior seems to be very random.
Note that the sequence of values passed to your comparator function is not, in general, defined. However, the sort function does all the necessary comparisons between any two values of the iterable it receives.
Is it creating a sorted list of keys internally where it keeps track of each comparison made?
No, it is not keeping a list of keys internally. Rather the sorting algorithm essentially iterates over the list you give it. In fact it builds subsets of lists to avoid doing too many comparisons - there is a nice visualization of how the sorting algorithm works at Visualising Sorting Algorithms: Python's timsort by Aldo Cortesi
Basically, for the simple list such as [2, 4, 6, 3, 1] and the complex list you provided, the sorting algorithms are the same.
The only differences are the complexity of elements in the list and the comparing scheme that how to compare any tow elements (e.g. myComparator you provided).
There is a good description for Python Sorting: https://wiki.python.org/moin/HowTo/Sorting
First, the cmp() function:
cmp(...)
cmp(x, y) -> integer
Return negative if x<y, zero if x==y, positive if x>y.
You are using this line: items.sort(myComparator) which is equivalent to saying: items.sort(-1) or items.sort(0) or items.sort(1)
Since you want to sort based on the sum of each tuples list, you could do this:
mylist = [(2, [3, 4, 5]), (3, [1, 0, 0, 0, 1]), (4, [-1]), (10, [1, 2, 3])]
sorted(mylist, key=lambda pair: sum(pair[1]))
What this is doing is, I think, exactly what you wanted. Sorting mylist based on the sum() of each tuples list
Is there a concept of range in Haskell or other functional programming languages? Suppose I want to compute I_AMAX from BLAS (to find the smallest index of the largest element in vector x). A naive method would be something like
idamax x idx = foldr(imax) 0 (zip x idx)
where idx is the index vector and imax returns the tuple with the smallest index and largest value.
For example, some vector (say, [1, 7, 3, 5]), zipped with its index vector ([0, 1, 2, 3], which makes [(1, 0), (7, 1), (3, 2), (5, 2)]), then reduced with the maximum function (so (7, 1), or 7).
But the index knowledge should be implicit, since in a loop the index is self-evident. Is there a way to write this in Haskell?
tl;dr, what's the best way to write I_AMAX in Haskell?
Here's a version:
idamax :: Ord a => [a] -> Int
idamax = negate . snd . maximum . flip zip [0, -1..]
Explanation
First note that zipping to lists yields a list with length of the shortest input list. So we can use [0..] to attach natural indices to list elements:
zip [0..] [1, 7, 3, 5] = [(0, 1), (1, 7), (2, 3), (3, 5)]
So what we do is:
attach an (negated) index to each element in a list
find a pair with max value and least index (which is max here because of previous negation)
extract index from resulting pair
negate it back
For example, starting with [1, 4, 2, 4]:
attach index: [(1, 0), (4, -1), (2, -2), (4, -3)]
find maximum: (4, -1) (note that (4, -1) > (4, -3))
extract index: -1
negate: 1
Examples
>>> idamax [1..10]
9
>>> idamax [1, 3, 4, 2, 4]
2
>>> idamax "oh hello"
0
A multi-set is a set in which all the elements may not be unique.How to enumerate all the possible permutations among the set elements?
Generating all the possible permutations and then discarding the repeated ones is highly inefficient. Various algorithms exist to directly generate the permutations of a multiset in lexicographical order or other kind of ordering. Takaoka's algorithm is a good example, but probably that of Aaron Williams is better
http://webhome.csc.uvic.ca/~haron/CoolMulti.pdf
moreover, it has been implemented in the R package ''multicool''.
Btw, if you just want the total number of distinct permutations, the answer is the Multinomial coefficient:
e.g., if you have, say, n_a elements 'a', n_b elements 'b', n_c elements 'c',
the total number of distinct permutations is (n_a+n_b+n_c)!/(n_a!n_b!n_c!)
This is my translation of the Takaoka multiset permutations algorithm into Python (available here and at repl.it):
def msp(items):
'''Yield the permutations of `items` where items is either a list
of integers representing the actual items or a list of hashable items.
The output are the unique permutations of the items given as a list
of integers 0, ..., n-1 that represent the n unique elements in
`items`.
Examples
========
>>> for i in msp('xoxox'):
... print(i)
[1, 1, 1, 0, 0]
[0, 1, 1, 1, 0]
[1, 0, 1, 1, 0]
[1, 1, 0, 1, 0]
[0, 1, 1, 0, 1]
[1, 0, 1, 0, 1]
[0, 1, 0, 1, 1]
[0, 0, 1, 1, 1]
[1, 0, 0, 1, 1]
[1, 1, 0, 0, 1]
Reference: "An O(1) Time Algorithm for Generating Multiset Permutations", Tadao Takaoka
https://pdfs.semanticscholar.org/83b2/6f222e8648a7a0599309a40af21837a0264b.pdf
'''
def visit(head):
(rv, j) = ([], head)
for i in range(N):
(dat, j) = E[j]
rv.append(dat)
return rv
u = list(set(items))
E = list(reversed(sorted([u.index(i) for i in items])))
N = len(E)
# put E into linked-list format
(val, nxt) = (0, 1)
for i in range(N):
E[i] = [E[i], i + 1]
E[-1][nxt] = None
head = 0
afteri = N - 1
i = afteri - 1
yield visit(head)
while E[afteri][nxt] is not None or E[afteri][val] < E[head][val]:
j = E[afteri][nxt] # added to algorithm for clarity
if j is not None and E[i][val] >= E[j][val]:
beforek = afteri
else:
beforek = i
k = E[beforek][nxt]
E[beforek][nxt] = E[k][nxt]
E[k][nxt] = head
if E[k][val] < E[head][val]:
i = k
afteri = E[i][nxt]
head = k
yield visit(head)
sympy provides multiset_permutations.
from the doc:
>>> from sympy.utilities.iterables import multiset_permutations
>>> from sympy import factorial
>>> [''.join(i) for i in multiset_permutations('aab')]
['aab', 'aba', 'baa']
>>> factorial(len('banana'))
720
>>> sum(1 for _ in multiset_permutations('banana'))
60
There are O(1) (per permutation) algorithms for multiset permutation generation, for example, from Takaoka (with implementation)
Optimisation of smichr's answer, I unzipped the nxts to make the visit function more efficient with an accumulate() (the map() is faster than a list comprehension and it seemed shallow and pedantic to have to nest it in a second one with a constant index)
from itertools import accumulate
def msp(items):
def visit(head):
'''(rv, j) = ([], head)
for i in range(N):
(dat, j) = E[j]
rv.append(dat)
return(rv)'''
#print(reduce(lambda e,dontCare: (e[0]+[E[e[1]]],nxts[e[1]]),range(N),([],head))[0])
#print(list(map(E.__getitem__,accumulate(range(N-1),lambda e,N: nxts[e],initial=head))))
return(list(map(E.__getitem__,accumulate(range(N-1),lambda e,N: nxts[e],initial=head))))
u=list(set(items))
E=list(sorted(map(u.index,items)))
N=len(E)
nxts=list(range(1,N))+[None]
head=0
i,ai,aai=N-3,N-2,N-1
yield(visit(head))
while aai!=None or E[ai]>E[head]:
beforek=(i if aai==None or E[i]>E[aai] else ai)
k=nxts[beforek]
if E[k]>E[head]:
i=k
nxts[beforek],nxts[k],head = nxts[k],head,k
ai=nxts[i]
aai=nxts[ai]
yield(visit(head))
Here are the test results (the second has (13!/2!/3!/3!/4!)/10! = 143/144 times as many permutations but takes longer due to being more of a multiset, I suppose), mine seems 9% and 7% faster respectively:
cProfile.run("list(msp(list(range(10))))")
cProfile.run("list(msp([0,1,1,2,2,2,3,3,3,3,4,4,4]))")
original:
43545617 function calls in 28.452 seconds
54054020 function calls in 32.469 seconds
modification:
39916806 function calls in 26.067 seconds
50450406 function calls in 30.384 seconds
I have insufficient reputation to comment upon answers, but for an items input list, Martin Böschen's answer has time complexity the product of the factorials of the number of instances of each element value times greater, or
reduce(int.__mul__,map(lambda n: reduce(int.__mul__,range(1,n+1)),map(items.count,set(items))))
This can grow large quickly when computing large multisets with many occurrences. For instance, it will take 1728 times longer per permutation for my second example than my first.
You can reduce your problem to enumerate all permutations of a list. The typcial permutation generation algorithm takes a list and don't check if elements are equal. So you only need to generate a list out of your multiset, and feed it to your permutation generating algorithm.
For example, you have the multiset {1,2,2}.
You transform it to the list [1,2,2].
And generate all permutations, for example in python:
import itertools as it
for i in it.permutations([1,2,2]):
print i
And you will get the output
(1, 2, 2)
(1, 2, 2)
(2, 1, 2)
(2, 2, 1)
(2, 1, 2)
(2, 2, 1)
The problem is, that you get some permutations repeatedly. A simple solution would be just to filter them out:
import itertools as it
permset=set([i for i in it.permutations([1,2,2])])
for x in permset:
print x
Output:
(1, 2, 2)
(2, 2, 1)
(2, 1, 2)
I am reading Hutton's book, Programming in Haskell.
Here is a function:
pairs :: [a] -> [(a,a)]
pairs xs = zip xs (tail xs)
e.g.
>pairs [1,2,3,4]
>[(1,2),(2,3),(3,4)] --result
Problem is how to read this function ? from left to right?
I am confused how "tail" leave 1 element and then combine it with next element using "zip"
Since "tail" suppose to get all remaining elements from the list right?
I haven't read the book you mentioned, but I'll try to explain what I know.
You're right about the tail function returning everything except the first element of the list. Let's see how zip works,
zip [1, 2, 3, 4] [5, 6, 7, 8]
gives,
[(1, 5), (2, 6), (3, 7), (4, 8)]
Now, consider the output we need from the input we have, observe the transformation required from input to output,
[1, 2, 3, 4] -> [(1,2),(2,3),(3,4)]
From the above application of zip, we can see the output we need can be obtained by calling zip with,
zip [1, 2, 3] [2, 3, 4]
Now, from the docs on zip function, we can see that if the two given lists are of unequal length, the extra items in the longer list are discarded. So, we'd get the same result with,
zip [1, 2, 3, 4] [2, 3, 4]
in which the last 4 in the first input list would be discarded and we get the result we want.
This can be written in a function as,
pairs xs = zip xs (tail xs)
If it is something else you are confused about, do let me know.
zip takes 2 arguments. tail returns its argument with the first element removed, but does not modify its argument.Therefore [1, 2, 3, 4] gets zipped with [2, 3, 4].