creating a list of tuples based on successive items of initial list [duplicate] - python-3.x

I sometimes need to iterate a list in Python looking at the "current" element and the "next" element. I have, till now, done so with code like:
for current, next in zip(the_list, the_list[1:]):
# Do something
This works and does what I expect, but is there's a more idiomatic or efficient way to do the same thing?
Some answers to this problem can simplify by addressing the specific case of taking only two elements at a time. For the general case of N elements at a time, see Rolling or sliding window iterator?.

The documentation for 3.8 provides this recipe:
import itertools
def pairwise(iterable):
"s -> (s0, s1), (s1, s2), (s2, s3), ..."
a, b = itertools.tee(iterable)
next(b, None)
return zip(a, b)
For Python 2, use itertools.izip instead of zip to get the same kind of lazy iterator (zip will instead create a list):
import itertools
def pairwise(iterable):
"s -> (s0, s1), (s1, s2), (s2, s3), ..."
a, b = itertools.tee(iterable)
next(b, None)
return itertools.izip(a, b)
How this works:
First, two parallel iterators, a and b are created (the tee() call), both pointing to the first element of the original iterable. The second iterator, b is moved 1 step forward (the next(b, None)) call). At this point a points to s0 and b points to s1. Both a and b can traverse the original iterator independently - the izip function takes the two iterators and makes pairs of the returned elements, advancing both iterators at the same pace.
Since tee() can take an n parameter (the number of iterators to produce), the same technique can be adapted to produce a larger "window". For example:
def threes(iterator):
"s -> (s0, s1, s2), (s1, s2, s3), (s2, s3, 4), ..."
a, b, c = itertools.tee(iterator, 3)
next(b, None)
next(c, None)
next(c, None)
return zip(a, b, c)
Caveat: If one of the iterators produced by tee advances further than the others, then the implementation needs to keep the consumed elements in memory until every iterator has consumed them (it cannot 'rewind' the original iterator). Here it doesn't matter because one iterator is only 1 step ahead of the other, but in general it's easy to use a lot of memory this way.

Roll your own!
def pairwise(iterable):
it = iter(iterable)
a = next(it, None)
for b in it:
yield (a, b)
a = b

Starting in Python 3.10, this is the exact role of the pairwise function:
from itertools import pairwise
list(pairwise([1, 2, 3, 4, 5]))
# [(1, 2), (2, 3), (3, 4), (4, 5)]
or simply pairwise([1, 2, 3, 4, 5]) if you don't need the result as a list.

I’m just putting this out, I’m very surprised no one has thought of enumerate().
for (index, thing) in enumerate(the_list):
if index < len(the_list):
current, next_ = thing, the_list[index + 1]
#do something

Since the_list[1:] actually creates a copy of the whole list (excluding its first element), and zip() creates a list of tuples immediately when called, in total three copies of your list are created. If your list is very large, you might prefer
from itertools import izip, islice
for current_item, next_item in izip(the_list, islice(the_list, 1, None)):
print(current_item, next_item)
which does not copy the list at all.

Iterating by index can do the same thing:
#!/usr/bin/python
the_list = [1, 2, 3, 4]
for i in xrange(len(the_list) - 1):
current_item, next_item = the_list[i], the_list[i + 1]
print(current_item, next_item)
Output:
(1, 2)
(2, 3)
(3, 4)

I am really surprised nobody has mentioned the shorter, simpler and most importantly general solution:
Python 3:
from itertools import islice
def n_wise(iterable, n):
return zip(*(islice(iterable, i, None) for i in range(n)))
Python 2:
from itertools import izip, islice
def n_wise(iterable, n):
return izip(*(islice(iterable, i, None) for i in xrange(n)))
It works for pairwise iteration by passing n=2, but can handle any higher number:
>>> for a, b in n_wise('Hello!', 2):
>>> print(a, b)
H e
e l
l l
l o
o !
>>> for a, b, c, d in n_wise('Hello World!', 4):
>>> print(a, b, c, d)
H e l l
e l l o
l l o
l o W
o W o
W o r
W o r l
o r l d
r l d !

This is now a simple Import As of 16th May 2020
from more_itertools import pairwise
for current, next in pairwise(your_iterable):
print(f'Current = {current}, next = {nxt}')
Docs for more-itertools
Under the hood this code is the same as that in the other answers, but I much prefer imports when available.
If you don't already have it installed then:
pip install more-itertools
Example
For instance if you had the fibbonnacci sequence, you could calculate the ratios of subsequent pairs as:
from more_itertools import pairwise
fib= [1,1,2,3,5,8,13]
for current, nxt in pairwise(fib):
ratio=current/nxt
print(f'Curent = {current}, next = {nxt}, ratio = {ratio} ')

As others have pointed out, itertools.pairwise() is the way to go on recent versions of Python. However, for 3.8+, a fun and somewhat more concise (compared to the other solutions that have been posted) option that does not require an extra import comes via the walrus operator:
def pairwise(iterable):
a = next(iterable)
yield from ((a, a := b) for b in iterable)

A basic solution:
def neighbors( list ):
i = 0
while i + 1 < len( list ):
yield ( list[ i ], list[ i + 1 ] )
i += 1
for ( x, y ) in neighbors( list ):
print( x, y )

Pairs from a list using a list comprehension
the_list = [1, 2, 3, 4]
pairs = [[the_list[i], the_list[i + 1]] for i in range(len(the_list) - 1)]
for [current_item, next_item] in pairs:
print(current_item, next_item)
Output:
(1, 2)
(2, 3)
(3, 4)

code = '0016364ee0942aa7cc04a8189ef3'
# Getting the current and next item
print [code[idx]+code[idx+1] for idx in range(len(code)-1)]
# Getting the pair
print [code[idx*2]+code[idx*2+1] for idx in range(len(code)/2)]

Related

SymPy result Filtering

I was recently working on a CodeForce problem
So, I was using SymPy to solve this.
My code is :
from sympy import *
x,y = symbols("x,y", integer = True)
m,n = input().split(" ")
sol = solve([x**2 + y - int(n), y**2 + x - int(m)], [x, y])
print(sol)
What I wanted to do:
Filter only Positive and integer value from SymPy
Ex: If I put 14 28 in the terminal it will give me tons of result, but I just want it to show [(5, 3)]
I don't think that this is the intended way to solve the code force problem (I think you're just supposed to loop over the possible values for one of the variables).
I'll show how to make use of SymPy here anyway though. Your problem is a diophantine system of equations. Although SymPy has a diophantine solver it only works for individual equations rather than systems.
Usually the idea of using a CAS for something like this though is to symbolically find something like a general result that then helps you to write faster concrete numerical code. Here are your equations with m and n as arbitrary symbols:
In [62]: x, y, m, n = symbols('x, y, m, n')
In [63]: eqs = [x**2 + y - n, y**2 + x - m]
Using the polynomial resultant we can eliminate either x or y from this system to obtain a quartic polynomial for the remaining variable:
In [31]: py = resultant(eqs[0], eqs[1], x)
In [32]: py
Out[32]:
2 2 4
m - 2⋅m⋅y - n + y + y
While there is a quartic general formula that SymPy can use (if you use solve or roots here) it is too complicated to be useful for a problem like the one that you are describing. Instead though the rational root theorem tells us that an integer root for y must be a divisor of the constant term:
In [33]: py.coeff(y, 0)
Out[33]:
2
m - n
Therefore the possible values for y are:
In [64]: yvals = divisors(py.coeff(y, 0).subs({m:14, n:28}))
In [65]: yvals
Out[65]: [1, 2, 3, 4, 6, 7, 8, 12, 14, 21, 24, 28, 42, 56, 84, 168]
Since x is m - y**2 the corresponding values for x are:
In [66]: solve(eqs[1], x)
Out[66]:
⎡ 2⎤
⎣m - y ⎦
In [67]: xvals = [14 - yv**2 for yv in yvals]
In [68]: xvals
Out[68]: [13, 10, 5, -2, -22, -35, -50, -130, -182, -427, -562, -770, -1750, -3122, -7042, -28210]
The candidate solutions are then given by:
In [69]: candidates = [(xv, yv) for xv, yv in zip(xvals, yvals) if xv > 0]
In [70]: candidates
Out[70]: [(13, 1), (10, 2), (5, 3)]
From there you can test which values are solutions:
In [74]: eqsmn = [eq.subs({m:14, n:28}) for eq in eqs]
In [75]: [c for c in candidates if all(eq.subs(zip([x,y],c))==0 for eq in eqsmn)]
Out[75]: [(5, 3)]
The algorithmically minded will probably see from the above example how to make a much more efficient way of implementing the solver.
I've figured out the answer to my question ! At first, I was trying to filter the result from solve(). But there is an easy way to do this.
Pseudo code:
solve() gives the intersection point of both Parabolic Equations as a List
I just need to filter() the other types of values. Which in my case is <sympy.core.add.Add>
def rem(_list):
return list(filter(lambda v: type(v) != Add, _list))
Yes, You can also use type(v) == int
Final code:
from sympy import *
# the other values were <sympy.core.add.Add> type. So, I just defined a function to filterOUT these specific types from my list.
def rem(_list):
return list(filter(lambda v: type(v) != Add, _list))
x,y = symbols("x,y", integer = True, negative = False)
output = []
m,n = input().split(' ')
# I need to solve these 2 equations separately. Otherwise, my defined function will not work without loop.
solX = rem(solve((x+(int(n)-x**2)**2 - int(m)), x))
solY = rem(solve((int(m) - y**2)**2 + y - int(n), y))
if len(solX) == 0 or len(solY) == 0:
print(0)
else:
output.extend(solX) # using "Extend" to add multiple values in the list.
output.extend(solY)
print(int((len(output))/2)) # Obviously, result will come in pairs. So, I need to divide the length of the list by 2.
Why I used this way :
I tried to solve it by algorithmic way, but it still had some float numbers. I just wanted to skip the loop thing here again !
As sympy solve() has already found the values. So, I skipped the other way and focused on filtering !
Sadly, code force compiler shows a runtime error! I guess it can't import sympy. However, it works fine in VSCode.

How to make create two list with one generator in python

I am trying to create two separate lists from a base list with only one generator but do not know how to do it.
this is the idea, I am wondering if there is a way to create the list's b and c below while only looping through a once.
a = [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]
b = [x[:2] for x in a]
c = [x[2:] for x in a]
What I did before this was just use a for loop through a and constantly append x[:2], x[2:] to b, c with every iteration but after using timeit module I found that using a generator is actually faster, and so I moved on to using two separate generators but now after using timeit with the above python code it seems to be just as slow as before the generators. I suspect it is because I am iterating through the list a twice now.
So basically my question is, what is the most efficient way to create b and c given a two dimensional list, for my application the base list, a, are quite large and so I need it as efficient as possible.
TL;DR: I would suggest to keep using list comprehension and bench mark before making any optimizations (if you really think you need them).
I tried four ways:
Using loops:
def use_loop(a):
b = []
c = []
for item in a:
b.append(item[:2])
c.append(item[2:])
return (b,c)
Using list comprehension twice:
def use_comprehension(a):
b = [x[:2] for x in a]
c = [x[2:] for x in a]
return (b,c)
Using list comprehension with zip
def use_comprehension_with_zip(a):
data = [[], []]
b, c = zip(*[(x[:2], x[2:]) for x in a])
return (list(b),list(c))
Usings threads is overkill and it will definitely increase your time.
def get_shorter_list(a, index, ans):
if index == 0:
for item in a:
ans.append(item[:2])
else:
for item in a:
ans.append(item[2:])
def use_threads(a):
b = []
c = []
data = {0:b, 1:c}
threads = []
for x in range(2):
thread = threading.Thread(target = get_shorter_list, args=(a, x, data.get(x)))
thread.start()
threads.append(thread)
for thread in threads:
thread.join()
return (b,c)
Then I used timeit for the three methods
function_list = [
use_loop,
use_comprehension,
use_comprehension_with_zip,
use_threads
]
assert use_loop(a) == use_comprehension(a) == use_comprehension_with_zip(a) == use_threads(a)
for function in function_list:
print(f'Time taken by {function.__name__}: {timeit.timeit("function(a)", globals = globals(), number = 10000)}')
That did not show a huge degradation by using list comprehension which I always prefer for clarity, brevity and speed.
Time taken by use_loop: 0.01102178400469711
Time taken by use_comprehension: 0.011585937994823325
Time taken by use_comprehension_with_zip: 0.0187399349961197
Time taken by use_threads: 2.036599840997951

How to multiply 2 input lists in python

Please help me understand how to code the following task in Python using input
Programming challenge description:
Write a short Python program that takes two arrays a and b of length n
storing int values, and returns the dot product of a and b. That is, it returns
an array c of length n such that c[i] = a[i] · b[i], for i = 0,...,n−1.
Test Input:
List1's input ==> 1 2 3
List2's input ==> 2 3 4
Expected Output: 2 6 12
Note that the dot product is defined in mathematics to be the sum of the elements of the vector c you want to build.
That said, here is a possibiliy using zip:
c = [x * y for x, y in zip(a, b)]
And the mathematical dot product would be:
sum(x * y for x, y in zip(a, b))
If the lists are read from the keyboard, they will be read as string, you have to convert them before applying the code above.
For instance:
a = [int(s) for s in input().split(",")]
b = [int(s) for s in input().split(",")]
c = [x * y for x, y in zip(a, b)]
Using for loops and appending
list_c = []
for a, b in zip(list_a, list_b):
list_c.append(a*b)
And now the same, but in the more compact list comprehension syntax
list_c = [a*b for a, b in zip(list_a, list_b)]
From iPython
>>> list_a = [1, 2, 3]
>>> list_b = [2, 3, 4]
>>> list_c = [a*b for a, b in zip(list_a, list_b)]
>>> list_c
[2, 6, 12]
The zip function packs the lists together, element-by-element:
>>> list(zip(list_a, list_b))
[(1, 2), (2, 3), (3, 4)]
And we use tuple unpacking to access the elements of each tuple.
From fetching the input and using map & lambda functions to provide the result. If you may want to print the result with spaces between (not as list), use the last line
list1, list2 = [], []
list1 = list(map(int, input().rstrip().split()))
list2 = list(map(int, input().rstrip().split()))
result_list = list(map(lambda x,y : x*y, list1, list2))
print(*result_list)
I came out with two solutions. Both or them are the ones that are expected in a Python introductory course:
#OPTION 1: We use the concatenation operator between lists.
def dot_product_noappend(list_a, list_b):
list_c = []
for i in range(len(list_a)):
list_c = list_c + [list_a[i]*list_b[i]]
return list_c
print(dot_product_noappend([1,2,3],[4,5,6])) #FUNCTION CALL TO SEE RESULT ON SCREEN
#OPTION 2: we use the append method
def dot_product_append(list_a, list_b):
list_c = []
for i in range(len(list_a)):
list_c.append(list_a[i]*list_b[i])
return list_c
print(dot_product_append([1,2,3],[4,5,6])) #FUNCTION CALL TO SEE RESULT ON SCREEN
Just note that the first method requires that you cast the product of integers to be a list before you can concatenate it to list_c. You do that by using braces ([[list_a[i]*list_b[i]] instead of list_a[i]*list_b[i]). Also note that braces are not necessary in the last method, because the append method does not require to pass a list as parameter.
I have added the two function calls with the values you provided, for you to see that it returns the correct result. Choose whatever function you like the most.

How to get element from iterator object?

can anyone try to explain to me how the following code work? As I understand, unpack is like a, b, i = [1,2,3] but how the following code work to get x? I have try to debug if I have x = iter(collections.deque([1,2,3,4,5], maxlen=1))
<_collections._deque_iterator object at 0x01239>
import collections
x, = iter(collections.deque([1,2,3,4,5], maxlen=1))
Here's a simpler example
>>> x = [24]
>>> x
[24]
>>> x, = [24]
>>> x
24
>>> x, y = [24, 96]
>>> x
24
>>> y
96
It's equivalent to your example since if you do list(iter(collections.deque([1,2,3,4,5], maxlen=1))) it's just a list with one element, [5].
You're correct that this is doing unpacking. You could write it as (x,) so that it looks more like a tuple if just x, is confusing. The comma after x makes x refer to the first element of a tuple with one element.
Your code above produces x=5 for me, not an iterator object.
Usually with an iterator object, (like a generator) use of the next() works
Here's a short example with the fibonacci sequence. Using the yield produces a generator object.
def iterative_fib(n):
a,b = 0,1
i=1
while i<n:
a, b = b, a+b
# print(b)
i+=1
yield b
x = iterative_fib(50)
next(x) # 12586269025
I'm not 100% sure specifically in your case, but try using next because next expects an iterator. If this doesn't work, then maybe produce a code example that replicates your issue.
Docs for next() : https://docs.python.org/3/library/functions.html#next
Edit:
Seeing some other answers regarding unpacking lists, here are some other ways using *:
a = [1,2,3,4,5]
a,b,c,d,e = [1,2,3,4,5] # you already mentioned this one
a, *b = [1,2,3,4,5] # a =[1], b=[2, 3, 4, 5]
a, *b, c, d = [1,2,3,4,5] #a =[1], b=[2,3], c=[4], d=[5]
*a, b, c = [1,2,3,4,5] # a=[1,2,3], b=[4], c=[5]
#But this wont work:
a, *b, *c = [1,2,3,4,5] # SyntaxError: two starred expressions in assignment

Split/partition list based on invariant/hash?

I have a list [a1,21,...] and would like to split it based on the value of a function f(a).
For example if the input is the list [0,1,2,3,4] and the function def f(x): return x % 3,
I would like to return a list [0,3], [1,4], [2], since the first group all takes values 0 under f, the 2nd group take value 1, etc...
Something like this works:
return [[x for x in lst if f(x) == val] for val in set(map(f,lst))],
But it does not seem optimal (nor pythonic) since the inner loop unnecessarily scans the entire list and computes same f values of the elements several times.
I'm looking for a solution that would compute the value of f ideally once for every element...
If you're not irrationally ;-) set on a one-liner, it's straightforward:
from collections import defaultdict
lst = [0,1,2,3,4]
f = lambda x: x % 3
d = defaultdict(list)
for x in lst:
d[f(x)].append(x)
print(list(d.values()))
displays what you want. f() is executed len(lst) times, which can't be beat
EDIT: or, if you must:
from itertools import groupby
print([[pair[1] for pair in grp]
for ignore, grp in
groupby(sorted((f(x), x) for x in lst),
key=lambda pair: pair[0])])
That doesn't require that f() produce values usable as dict keys, but incurs the extra expense of a sort, and is close to incomprehensible. Clarity is much more Pythonic than striving for one-liners.
#Tim Peters is right, and here is a mentioned setdefault and another itertool.groupby option.
Given
import itertools as it
iterable = range(5)
keyfunc = lambda x: x % 3
Code
setdefault
d = {}
for x in iterable:
d.setdefault(keyfunc(x), []).append(x)
list(d.values())
groupby
[list(g) for _, g in it.groupby(sorted(iterable, key=keyfunc), key=keyfunc)]
See also more on itertools.groupby

Resources