Related
In the below program, I am trying to add all my "new_list" values into my "fin_list". But append is not working as excpected and is overwriting it with whatever the "new_list"value is in that particular loop.
def all_subsequences(ind, a, new_list, fin_list):
if(ind >= len(a)):
print(new_list)
fin_list.append(new_list)
return
new_list.append(a[ind])
all_subsequences(ind+1, a, new_list, fin_list)
#new_list.remove(new_list[len(new_list)-1])
new_list.pop()
all_subsequences(ind+1, a, new_list, fin_list)
return fin_list
a = [3,1,2]
new_list = []
final_list = []
result = all_subsequences(0, a, new_list, final_list)
print(result)
Here the output at each level is as below
[3, 1, 2], [3, 1], [3, 2], [3], [1, 2], [1], [2], []
Since the last value is an empty list the final list value at the last is as below
[[], [], [], [], [], [], [], []]
Link to python sandbox :-
https://pythonsandbox.com/code/pythonsandbox_u21270_9PqNjYIsl7M85NGf4GBSLLrW_v0.py
I have tried to use extend instead of append inside the base condition but that is not the kind of result i am looking for. I am open to any suggestion to resolve this problem.
When you call fin_list.append(new_list), you are appending the reference of new_list to fin_list instead of copying fin_list. Therefore, when you do new_list.pop() later, if you print fin_list, you will find it's also changed.
The situation can be illustrated by this example:
foo = [1, 2, 3]
bar = []
bar.append(foo)
print(f"bar: {bar}")
# modify foo and you will find that bar is also modified
foo.append(4)
print(f"bar: {bar}")
The simplest way to solve the problem is to use fin_list.append(new_list[:]), which will copy new_list and append the copy to fin_list.
def all_subsequences(ind, a, new_list, fin_list):
if (ind >= len(a)):
print(new_list)
fin_list.append(new_list[:])
return
new_list.append(a[ind])
all_subsequences(ind+1, a, new_list, fin_list)
new_list.pop()
all_subsequences(ind+1, a, new_list, fin_list)
return fin_list
a = [3, 1, 2]
new_list = []
final_list = []
result = all_subsequences(0, a, new_list, final_list)
print(result)
I came across a weird syntactical approach at work today that I couldn't wrap my head around. Let's say I have the following list:
my_list = [[1, 2, 3], [4, 5, 6]]
My objective is to filter each nested list according to some criteria and overwrite the elements of the list in place. So, let's say I want to remove odd numbers from each nested list such that my_list contains lists of even numbers, where the end result would look like this:
[[2], [4, 6]]
If I try to do this using a simple assignment operator, it doesn't work.
my_list = [[1, 2, 3], [4, 5, 6]]
for l in my_list:
l = [num for num in l if num % 2 == 0]
print(my_list)
Output: [[1, 2, 3], [4, 5, 6]]
However, if I "slice" the list, it provides the expected output.
my_list = [[1, 2, 3], [4, 5, 6]]
for l in my_list:
l[:] = [num for num in l if num % 2 == 0]
print(my_list)
Output: [[2], [4, 6]]
My original hypothesis was that l was a newly created object that didn't actually point to the corresponding object in the list, but comparing the outputs of id(x[i]), id(l), and id(l[:]) (where i is the index of l in x), I realized that l[:] was the one with the differing id. So, if Python is creating a new object when I assign to l[:] then how does Python know to overwrite the existing object of l? Why does this work? And why doesn't the simple assignment operator l = ... work?
It's subtle.
Snippet one:
my_list = [[1, 2, 3], [4, 5, 6]]
for l in my_list:
l = [num for num in l if num % 2 == 0]
Why doesn't this work? Because when you do l = , you're only reassigning the variable l, not making any change to its value.
If we write the loop out "manually", it hopefully will become more clear why this strategy fails:
my_list = [[1, 2, 3], [4, 5, 6]]
# iteration 1
l = my_list[0]
l = [num for num in l if num % 2 == 0]
# iteration 2
l = my_list[1]
l = [num for num in l if num % 2 == 0]
Snippet two:
my_list = [[1, 2, 3], [4, 5, 6]]
for l in my_list:
l[:] = [num for num in l if num % 2 == 0]
Why does this work? Because by using l[:] = , you're actually modifying the value that l references, not just the variable l. Let me elaborate.
Generally speaking, using [:] notation (slice notation) on lists allows one to work with a section of the list.
The simplest use is for getting values out of a list; we can write a[n:k] to get the nth, item n+1st item, etc, up to k-1. For instance:
>>> a = ["a", "very", "fancy", "list"]
>>> print(a[1:3])
['very', 'fancy']
Python also allows use of slice notation on the left-side of a =. In this case, it interprets the notation to mean that we want to update only part of a list. For instance, we can replace "very", "fancy" with "not", "so", "fancy" like so:
>>> print(a)
['a', 'very', 'fancy', 'list']
>>> a[1:3] = ["not", "so", "fancy"]
>>> print(a)
['a', 'not', 'so', 'fancy', 'list']
When using slice syntax, Python also provides some convenient shorthand. Instead of writing [n:k], we can omit n or k or both.
If we omit n, then our slice looks like [:k], and Python understands it to mean "up to k", i.e., the same as [0:k].
If we omit k, then our slice looks like a[n:], and Python understands it to mean "n and after", i.e., the same as a[n:len(a)].
If we omit both, then both rules take place, so a[:] is the same as a[0:len(a)], which is a slice over the entire list.
Examples:
>>> print(a)
['a', 'not', 'so', 'fancy', 'list']
>>> print(a[2:4])
['so', 'fancy']
>>> print(a[:4])
['a', 'not', 'so', 'fancy']
>>> print(a[2:])
['so', 'fancy', 'list']
>>> print(a[:])
['a', 'not', 'so', 'fancy', 'list']
Crucially, this all still applies if we are using our slice on the left-hand side of a =:
>>> print(a)
['a', 'not', 'so', 'fancy', 'list']
>>> a[:4] = ["the", "fanciest"]
>>> print(a)
['the', 'fanciest', 'list']
And using [:] means to replace every item in the list:
>>> print(a)
['the', 'fanciest', 'list']
>>> a[:] = ["something", "completely", "different"]
>>> print(a)
['something', 'completely', 'different']
Okay, so far so good.
They key thing to note is that using slice notation on the left-hand side of a list updates the list in-place. In other words, when I do a[1:3] =, the variable a is never updated; the list that it references is.
We can see this with id(), as you were doing:
>>> print(a)
['something', 'completely', 'different']
>>> print(id(a))
139848671387072
>>> a[1:] = ["truly", "amazing"]
>>> print(a)
['something', 'truly', 'amazing']
>>> print(id(a))
139848671387072
Perhaps more pertinently, this means that if a were a reference to a list within some other object, then using a[:] = will update the list within that object. Like so:
>>> list_of_lists = [ [1, 2], [3, 4], [5, 6] ]
>>> second_list = list_of_lists[1]
>>> print(second_list)
[3, 4]
>>> second_list[1:] = [2, 1, 'boom!']
>>> print(second_list)
[3, 2, 1, 'boom!']
>>> print(list_of_lists)
[[1, 2], [3, 2, 1, 'boom!'], [5, 6]]
I'm trying to use list comprehension to replace a for loop for the purpose of improving speed. Being new to list comprehensions, it appears that I do not have a complete grasp of how they work. My intent is to pass a list to a function that is held in memory to complete the processing that needs to be done, then return the list. In this simplified example, it looks like instead of returning the list I want it is returning a list of identical lists. So instead of [1, 2, 3, 4, 5] I get [[1, 2, 3, 4, 5], [1, 2, 3, 4, 5], [1, 2, 3, 4, 5], [1, 2, 3, 4, 5], [1, 2, 3, 4, 5]]. I looked at the result here but it looks like they had a different problem.
lst1 = [1,2,3,4,5]
k = []
def main():
lst2 = [processfile(x) for x in lst1]
print(lst2)
def processfile(i):
k.append(i)
return(k)
if __name__ == '__main__':
main()
Any help would be greatly appreciated.
If you want to return a list then , you don't need to append the items in the lst1 to k just return i
lst1 = [1,2,3,4,5]
def main():
lst2 = [processfile(x) for x in lst1]
print(lst2)
def processfile(i):
return(i)
if __name__ == '__main__':
main()
Since k is only being used in the process file() function, avoid declaring it in the global scope. I didn't quite understand what you wanted/tried to do, but I hope this example will help you with list comprehension, also, I'd recommend you add 'Python' to the title of your post.
Here I will use list comprehension to add 1 to all items in list1.
lst1 = [1,2,3,4,5]
def main():
lst2 = [add1(item) for item in lst1]
print(lst2)
def add1(num):
return num + 1
if __name__ == '__main__':
main()
Expected output is lst2 = [2,3,4,5,6]
Just
lst2 = [processfile(x) for x in lst1]
will be fine.
In that case you should do like below:
def processfile(i):
return(i)
I have a list ['','','',['',[['a','b']['c']]],[[['a','b'],['c']]],[[['d']]]]
I want to flatten the list with indices and the output should be as follows:
flat list=['','','','','a','b','c','a','b','c','d']
indices=[0,1,2,3,3,3,3,4,4,4,5]
How to do this?
I have tried this:
def flat(nums):
res = []
index = []
for i in range(len(nums)):
if isinstance(nums[i], list):
res.extend(nums[i])
index.extend([i]*len(nums[i]))
else:
res.append(nums[i])
index.append(i)
return res,index
But this doesn't work as expected.
TL;DR
This implementation handles nested iterables with unbounded depth:
def enumerate_items_from(iterable):
cursor_stack = [iter(iterable)]
item_index = -1
while cursor_stack:
sub_iterable = cursor_stack[-1]
try:
item = next(sub_iterable)
except StopIteration:
cursor_stack.pop()
continue
if len(cursor_stack) == 1:
item_index += 1
if not isinstance(item, str):
try:
cursor_stack.append(iter(item))
continue
except TypeError:
pass
yield item, item_index
def flat(iterable):
return map(list, zip(*enumerate_items_from(a)))
Which can be used to produce the desired output:
>>> nested = ['', '', '', ['', [['a', 'b'], ['c']]], [[['a', 'b'], ['c']]], [[['d']]]]
>>> flat_list, item_indexes = flat(nested)
>>> print(item_indexes)
[0, 1, 2, 3, 3, 3, 3, 4, 4, 4, 5]
>>> print(flat_list)
['', '', '', '', 'a', 'b', 'c', 'a', 'b', 'c', 'd']
Note that you should probably put the index first to mimic what enumerate does. It would be easier to use for people that already know enumerate.
Important remark unless you are certain your lists will not be nested too much, you shouldn't use any recursion-based solution. Otherwise as soon as you'll have a nested list with depth greater than 1000, your code will crash. I explain this here. Note that a simple call to str(list) will crash on a test case with depth > 1000 (for some python implementations it's more than that, but it's always bounded). The typical exception you'll have when using recursion-based solutions is (this in short is due to how python call stack works):
RecursionError: maximum recursion depth exceeded ...
Implementation details
I'll go step by step, first we will flatten a list, then we will output both the flattened list and the depth of all items, and finally we will output both the list and the corresponding item indexes in the "main list".
Flattening list
That being said, this is actually quite interesting as the iterative solution is perfectly designed for that, you can take a simple (non-recursive) list flattening algorithm:
def flatten(iterable):
return list(items_from(iterable))
def items_from(iterable):
cursor_stack = [iter(iterable)]
while cursor_stack:
sub_iterable = cursor_stack[-1]
try:
item = next(sub_iterable)
except StopIteration: # post-order
cursor_stack.pop()
continue
if isinstance(item, list): # pre-order
cursor_stack.append(iter(item))
else:
yield item # in-order
Computing depth
We can have access to the depth by looking at the stack size, depth = len(cursor_stack) - 1
else:
yield item, len(cursor_stack) - 1 # in-order
This will return an iterative on pairs (item, depth), if we need to separate this result in two iterators we can use the zip function:
>>> a = [1, 2, 3, [4 , [[5, 6], [7]]], [[[8, 9], [10]]], [[[11]]]]
>>> flatten(a)
[(1, 0), (2, 0), (3, 0), (4, 1), (5, 3), (6, 3), (7, 3), (8, 3), (9, 3), (10, 3), (11, 3)]
>>> flat_list, depths = zip(*flatten(a))
>>> print(flat_list)
(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
>>> print(depths)
(0, 0, 0, 1, 3, 3, 3, 3, 3, 3, 3)
We will now do something similar to have item indexes instead of the depth.
Computing item indexes
To instead compute item indexes (in the main list), you'll need to count the number of items you've seen so far, which can be done by adding 1 to an item_index every time we iterate over an item that is at depth 0 (when the stack size is equal to 1):
def flatten(iterable):
return list(items_from(iterable))
def items_from(iterable):
cursor_stack = [iter(iterable)]
item_index = -1
while cursor_stack:
sub_iterable = cursor_stack[-1]
try:
item = next(sub_iterable)
except StopIteration: # post-order
cursor_stack.pop()
continue
if len(cursor_stack) == 1: # If current item is in "main" list
item_index += 1
if isinstance(item, list): # pre-order
cursor_stack.append(iter(item))
else:
yield item, item_index # in-order
Similarly we will break pairs in two itératifs using ˋzip, we will also use ˋmap to transform both iterators to lists:
>>> a = [1, 2, 3, [4 , [[5, 6], [7]]], [[[8, 9], [10]]], [[[11]]]]
>>> flat_list, item_indexes = map(list, zip(*flatten(a)))
>>> print(flat_list)
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
>>> print(item_indexes)
[0, 1, 2, 3, 3, 3, 3, 4, 4, 4, 5]
improvement — Handling iterable inputs
Being able to take a broader palette of nested iterables as input could be desirable (especially if you build this for others to use). For example, the current implementation doesn't work as expected if we have nested iterables as input, for example:
>>> a = iter([1, '2', 3, iter([4, [[5, 6], [7]]])])
>>> flat_list, item_indexes = map(list, zip(*flatten(a)))
>>> print(flat_list)
[1, '2', 3, <list_iterator object at 0x100f6a390>]
>>> print(item_indexes)
[0, 1, 2, 3]
If we want this to work we need to be a bit careful because strings are iterable but we want them to be considered as atomic items (not a as lists of characters). Instead of assuming the input is a list as we did before:
if isinstance(item, list): # pre-order
cursor_stack.append(iter(item))
else:
yield item, item_index # in-order
We will not inspect the input type, instead we will try to use it as if it was an iterable and if it fails we will know that it’s not an iterable (duck typing):
if not isinstance(item, str):
try:
cursor_stack.append(iter(item))
continue
# item is not an iterable object:
except TypeError:
pass
yield item, item_index
With this implementation, we have:
>>> a = iter([1, 2, 3, iter([4, [[5, 6], [7]]])])
>>> flat_list, item_indexes = map(list, zip(*flatten(a)))
>>> print(flat_list)
[1, 2, 3, 4, 5, 6, 7]
>>> print(item_indexes)
[0, 1, 2, 3, 3, 3, 3]
Building test cases
If you need to generate tests cases with large depths, you can use this piece of code:
def build_deep_list(depth):
"""Returns a list of the form $l_{depth} = [depth-1, l_{depth-1}]$
with $depth > 1$ and $l_0 = [0]$.
"""
sub_list = [0]
for d in range(1, depth):
sub_list = [d, sub_list]
return sub_list
You can use this to make sure my implementation doesn't crash when the depth is large:
a = build_deep_list(1200)
flat_list, item_indexes = map(list, zip(*flatten(a)))
We can also check that we can't print such a list by using the str function:
>>> a = build_deep_list(1200)
>>> str(a)
RecursionError: maximum recursion depth exceeded while getting the repr of an object
Function repr is called by str(list) on every element from the input list.
Concluding remarks
In the end I agree that recursive implementations are way easier to read (as the call stack does half the hard work for us), but when implementing low level function like that I think it is a good investment to have a code that works in all cases (or at least all the cases you can think of). Especially when the solution is not that hard. That's also a way not to forget how to write non-recursive code working on tree-like structures (which may not happen a lot unless you are implementing data structures yourself, but that's a good exercise).
Note that everything I say “against” recursion is only true because python doesn't optimize call stack usage when facing recursion: Tail Recursion Elimination in Python. Whereas many compiled languages do Tail Call recursion Optimization (TCO). Which means that even if you write the perfect tail-recursive function in python, it will crash on deeply nested lists.
If you need more details on the list flattening algorithm you can refer to the post I linked.
Simple and elegant solution:
def flat(main_list):
res = []
index = []
for main_index in range(len(main_list)):
# Check if element is a String
if isinstance(main_list[main_index], str):
res.append(main_list[main_index])
index.append(main_index)
# Check if element is a List
else:
sub_list = str(main_list[main_index]).replace('[', '').replace(']', '').replace(" ", '').replace("'", '').split(',')
res += sub_list
index += ([main_index] * len(sub_list))
return res, index
this does the job, but if you want it to be just returned then I'll enhance it for you
from pprint import pprint
ar = ["","","",["",[["a","b"],["c"]]],[[["a","b"],["c"]]],[[["d"]]]]
flat = []
indices= []
def squash(arr,indx=-1):
for ind,item in enumerate(arr):
if isinstance(item, list):
squash(item,ind if indx==-1 else indx)
else:
flat.append(item)
indices.append(ind if indx==-1 else indx)
squash(ar)
pprint(ar)
pprint(flat)
pprint(indices)
EDIT
and this is if you don't want to keep the lists in memory and return them
from pprint import pprint
ar = ["","","",["",[["a","b"],["c"]]],[[["a","b"],["c"]]],[[["d"]]]]
def squash(arr,indx=-1,fl=[],indc=[]):
for ind,item in enumerate(arr):
if isinstance(item, list):
fl,indc = squash(item,ind if indx==-1 else indx, fl, indc)
else:
fl.append(item)
indc.append(ind if indx==-1 else indx)
return fl,indc
flat,indices = squash(ar)
pprint(ar)
pprint(flat)
pprint(indices)
I'm not expecting you would need more than 1k recursion depth which is the default setting
Okay, so I have this function that I need to create and I think the code checker is somehow flawed and I tried manage it but my code still seems to fail
def reversecomp(L):
""" assumes L is a list of lists whose elements are ints
Mutates L such that it reverses its elements and also
reverses the order of the int elements in every element of L.
It does not return anything.
"""
if L == []:
return L
elif type(L) == int:
return L
else:
return reversecomp(L[1:]) + [reversecomp(L[0])]
def run_code(L):
return reversecomp(L)
print(L)
The question states that you need to mutate L. Your code must work when you do this:
L = [[0, 1, 2], [1, 2, 3], [3, 2, 1], [10, -10, 100]]
reversecomp(L)
print(L)
Test: run_code([[0, 1, 2], [1, 2, 3]])
Your output:
[[3, 2, 1], [2, 1, 0]]
Correct output:
[[3, 2, 1], [2, 1, 0]]
None
The spec says "It does not return anything"; your program does.
L is a list of lists of ints
Okay, so why are you checking type(L) == int when type(L) == list is always true, per the specification?
Mutates L
You're not mutating L at all; you're returning a new list. Mutating L means doing something like L[...] = xxx.
It does not return anything.
You shouldn't be using the return keyword at all in reversecomp.