I'm trying to construct permutations of a list with the i-th bit flipped every time.
For example:
With input:
[1,0,0,0]
To get:
[0,0,0,0]
[1,1,0,0]
[1,0,1,0]
[1,0,0,1]
I wrote a method which given a list returns the same list with the bit at position p changed:
def flipBit(l,p):
l[p] = ~l[p]&1
return l
And I'm trying to apply it using a map(), but I can't even get a basic example working:
p=list(map(flipBit, [[1,0,0]]*3,range(3))))
This is what it returns:
[[0, 1, 1], [0, 1, 1], [0, 1, 1]]
while expecting:
[[0, 0, 0], [1, 1, 0], [1, 0, 1]]
What I'm I doing wrong? (if anyone can suggest an even shorter code for this maybe without using a flipbit method I'd appreciate it as I won't really use flipbit other than this purpose and I want to keep the code concise and clean)
The issue is that [[1,0,0]]*3 creates a list containing three references to the same sublist. When you change one sublist, they all change.
Here is one way to fix this:
>>> list(map(flipBit, [[1,0,0] for _ in range(3)], range(3)))
↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑
[[0, 0, 0], [1, 1, 0], [1, 0, 1]]
And here is a way to implement this functionality without using a helper function:
>>> l = [1, 0, 0]
>>> [l[:i] + [1-l[i]] + l[i+1:] for i in range(len(l))]
[[0, 0, 0], [1, 1, 0], [1, 0, 1]]
The code you posted is was not valid at all, but I presume you need this:
>>> p=list(map(lambda x: flipBit([1, 0, 0], x), range(3)))
>>> p
[[0, 0, 0], [1, 1, 0], [1, 0, 1]]
Basically, you map with a lambda function, that partially applies [1, 0, 0] as l, and then takes each element in range(3) and applies it to p.
Related
I am trying to improve a part of code that is slowing down the whole script significantly, right to the point of making it unfeasible. In particular the piece of code is:
for vectors1 in EC1:
for vectors2 in EC2:
r = np.add(vectors1, vectors2)
for vectors3 in CDC:
result = np.add(r, vectors3).tolist()
if result not in states: # This is what makes it very slow
states.append(result)
EC1, EC2 and CDC are lists that contains as elements, lists of lists, as an example of one iteration, we get:
vectors1: [[2, 0, 0], [0, 0, 0], [0, 0, 0], [2, 0, 0], [0, 0, 0], [0, 0, 0], [2, 0, 0], [2, 0, 0], [0, 0, 0]]
vectors2: [[0, 0, 0], [2, 0, 0], [0, 0, 0], [0, 0, 0], [2, 0, 0], [2, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0]]
vectors3: [[0, 0, 0], [0, 0, 0], [2, 1, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [2, 1, 0], [2, 1, 0]]
result: [[2, 0, 0], [2, 0, 0], [2, 1, 0], [2, 0, 0], [2, 0, 0], [2, 0, 0], [2, 0, 0], [4, 1, 0], [2, 1, 0]]
Notice how vectors1, vectors2 and vectors3 correspond to one element from EC1, EC2 and CDC respectively, also how 'result' is the summation from vectors1, vectors2 and vectors3, hence the previous vectors cannot be altered in any manner or sorted, otherwise it would change the expected result from the 'result' variable.
In the first two loops each item in EC1 and EC2 are summed, for later on sum up the previous result with items in CDC. To sum the list of lists from EC1 and EC2 and later on the previous result ('r') with the list of lists from CDC I use numpy.add(). Finally, I reconvert 'result' back to list. So Basically I am managing lists of lists as elements from EC1, EC2 and CDC.
The problem is that I must deal with hundreds of thousands (close to 1M) of results and having to check if a result exists in states list is slowing things drastically, specially since states list grows as more results are processed.
I've tried to keep inside the numpy world by managing everything as numpy arrays. First declaring states as:
states = np.empty([9, 3], int)
Then, concatenating the result numpy array to states numpy array, prior checking if already exists in states:
for vectors1 in EC1:
for vectors2 in EC2:
r = np.add(vectors1, vectors2)
for vectors3 in CDC:
result = np.add(r, vectors3)
if not np.isin(states, result).any():
np.concatenate(states, result, axis=0)
But definitely I am doing something wrong because result is not being concatenated to states, I've also tried without success:
np.append(states, result, axis=0)
Could this be parallelized in some way?
You can do the sums solely in numpy by using broadcasting
res = ((EC1[:,None,:] + EC2).reshape(-1, 1, 3) + CDC).reshape(-1, 3)
given that EC1, EC2 and CDC are arrays.
Afterwards you can filter out the duplicates with
np.unique(res, axis=0)
But like Lucas, I would strongly advise you to filter the arrays beforehand. For your example arrays that would shrink the number of rows in res from 729 to 8.
I'm not sure how large the data are that you are working with but this may speed things up somewhat:
EC1 = [[2, 0, 0], [0, 0, 0], [0, 0, 0], [2, 0, 0], [0, 0, 0], [0, 0, 0], [2, 0, 0], [2, 0, 0], [0, 0, 0]]
EC2 = [[0, 0, 0], [2, 0, 0], [0, 0, 0], [0, 0, 0], [2, 0, 0], [2, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0]]
CDC = [[0, 0, 0], [0, 0, 0], [2, 1, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [2, 1, 0], [2, 1, 0]]
EC1.sort()
EC2.sort()
CDC.sort()
unique_triples = dict()
for v1 in EC1:
for v2 in EC2:
for v3 in CDC:
if str(v1)+str(v2)+str(v3) not in unique_triples: # list not hashable but strings are
unique_triples[str(v1)+str(v2)+str(v3)] = list(np.add(np.add(v1, v2), v3))
The basic idea is to remove duplicate triples of (EC1,EC2, CDC) entries and only do the additions on unique triples, sort the lists so that they are ordered lexicographically
A dictionary has O(1) lookups so these lookups are (maybe) faster.
Whether this is faster or not might depend on how large-and how many unique values of triples-the data are that are being processed.
The 3-vector sums are the values of the dictionary, e.g.
list(unique_triples.values()) for me gives:
>>> list(unique_triples.values())
[[0, 0, 0], [2, 1, 0], [2, 0, 0], [4, 1, 0], [2, 0, 0], [4, 1, 0], [4, 0, 0], [6, 1, 0]]
I did not remove the duplicates in the original lists of lists here. If the application you are looking at allows, it is also likely beneficial to remove these duplicates in EC1, EC2, and CDC before iterating over the values.
Expected Output:
indenitiy_matrix(3)
[[1, 0, 0], [0, 1, 0], [0, 0, 1]]
Actual Output with Error:
indenitiy_matrix(3)
[[1, 1, 1], [1, 1, 1], [1, 1, 1]]
def identity_matrix(n):
list_template = [[]]
list_n = list_template*n
for sub_l in list_n:
sub_l.append(0)
for val in range(n):
# I have the feeling that the problem lies somewhere around here.
list_n[val][val]=1
return(list_n)
list_template*n does not create n copies, instead but all those n copies reference to only one copy. For example see this
a = [[0,0,0]]*2
# Now, lets change first element of the first sublist in `a`.
a[0][0] = 1
print (a)
# but since both the 2 sublists refer to same, both of them will be changed.
Output:
[[1, 0, 0], [1, 0, 0]]
Fix for your code
def identity_matrix(n):
list_n = [[0]*n for i in range(n)]
for val in range(n):
list_n[val][val]=1
return list_n
print (identity_matrix(5))
Output:
[[1, 0, 0, 0, 0],
[0, 1, 0, 0, 0],
[0, 0, 1, 0, 0],
[0, 0, 0, 1, 0],
[0, 0, 0, 0, 1]]
No, the problem lies here:
list_template = [[]]
list_n = list_template*n
After this, try doing:
list_n[0].append(1) # let's change the first element
The result:
[[1], [1], [1], [1], [1]]
is probably not what you expect.
Briefly, the problem is that after its construction, your list consists of multiple references to same list. A detailed explanation is at the link given by #saint-jaeger : List of lists changes reflected across sublists unexpectedly
Finally, the numpy library is your friend for creating identity matrices and other N-dimensional arrays.
I have a function that will display numbers on a "grid". The rows need to be printed from the bottom up, and the columns to printed from left to right.
Ex.
801
000
402
[[4, 0, 8], [0, 0, 0], [2, 0, 1]]
My current code is printing out
201
000
408
[[4, 0, 8], [0, 0, 0], [2, 0, 1]]
The row 4 0 8 should print UP in the first column.
def display(grid):
for rows in range(len(grid)-1, -1, -1):
#for rows in grid[::-1]:
for columns in grid[rows]: # need to change the columns
print(columns, end="")
print("")
print("\n")
return
Using numpy:
numpy.array(lst).transpose()[::-1]
Outputs in:
array([[8, 0, 1],
[0, 0, 0],
[4, 0, 2]])
import itertools
lst = [[8, 0, 1], [0, 0, 0], [4, 0, 2]]
lst1 = list(itertools.chain.from_iterable(lst)) #unpack all sublist using iter chain
final = [[x for x in lst1[-3::-3]], [x for x in lst1[-2::-3]], [x for x in
lst1[-1::-3]]] # formulate the desired list using lsit comprehesion
print(final)
Result
[[4, 0, 8], [0, 0, 0], [2, 0, 1]]
Hope you can help me with this:
previous=[[0, 0, 1, 1], [0, 0, 0, 0], [0, 0, 1, 2], [0, 0, 1, 2], [0, 0, 1, 1], [0, 1, 1, 2], [0, 1, 1, 1], [0, 0, 1, 2], [0, 1, 1, 2], [0, 0, 0, 0]]
type(previous)
Out[67]: list
previous[0][-1]
Out[66]: 1
previous[1][-1]
Out[65]: 0
for p in previous:
print(p)
[0, 0, 1, 1]
[0, 0, 0, 0]
[0, 0, 1, 2]
[0, 0, 1, 2]
[0, 0, 1, 1]
[0, 1, 1, 2]
[0, 1, 1, 1]
[0, 0, 1, 2]
[0, 1, 1, 2]
[0, 0, 0, 0]
My Question, I need to get the 3 value from each element in the sub list and append it to create another list.
The result should be:
[1,0,1,1,1,1,1,1,1,0]
I tried this but without success:
mylist=[]
for p in previous:
for x in p:
mylist.append(p[x][-1])
print(mylist)
and:
for p in previous:
for x in p:
print ([p[x]])
p is the sublist, you don't need to loop over it again with for x in p: to get the third element (so your code actually raises an IndexError when trying to index the sublist p if x is out of its bound, and will raise an TypeError when trying to use p[x][-1] to index the number at p[x]), still, p[-1] is the last element (the fourth in your case), you should either use positive indexing:
p[2] # the third element
Or use negative indexing to get the next-to-last element:
p[-2]
But if the lists should grow any longer, positive indexing is better since it won't be affected by elements added to the end of the list.
You could also write a list comprehension instead of a for loop + append:
mylist = [p[2] for p in previous]
print(mylist)
Output:
[1, 0, 1, 1, 1, 1, 1, 1, 1, 0]
In your loop p is the sublist. The first time through the loop p is
[0, 0, 1, 1]
Python does zero based indexing (the first element is always at index 0). So p[0] is the first element. The third element would be p[2].
If you use negative indexes, -1 gives the last element and -2 is the next-to-last or penultimate element.
To modify your solution to work, you can remove the inner loop. You don't need it. You already have the sublist.
mylist=[]
for p in previous:
mylist.append(p[2])
print(mylist)
I had a confusion where I initialize a 2D array in this way:
>>> a = [[0] * 3] * 3
>>> a
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]
>>> a[0][0] = 1
>>> a
[[1, 0, 0], [1, 0, 0], [1, 0, 0]]
while I was expecting it to be
[[1, 0, 0], [0, 0, 0], [0, 0, 0]]
I also tried
>>> a = [[0 for _ in range(3)] for _ in range(3)]
>>> a
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]
>>> a[0][0] = 1
>>> a
[[1, 0, 0], [0, 0, 0], [0, 0, 0]]
Which worked as expected
Wonder what is the reason caused this?
With the outter *3 you are making a shallow copy of a list.
This can be easily verifier by printing ids. An id is unique to each item.
a = [[0] * 3] * 3
print(*map(id, a))
# Same ID
Instead, you absolutely need to generate new lists
a = [[0] * 3 for _ in range(3)]
print(*map(id, a))
# Different ids
If you want more information you can check this question : What exactly is the difference between shallow copy, deepcopy and normal assignment operation?