how to operate None in the list comprehension? - python-3.x

I want to make [2,3,None,4,None] into [4,6,None,8,None],how can i do?
>>> v= [2,3,None,4,None]
>>> [x*2 for x in v if not x is None]
[4, 6, 8]

You're almost there. Just add an else clause to include the Nones:
>>> [x*2 if x is not None else x for x in v]
[4, 6, None, 8, None]
Note that you must put the if-else before the for x in v; otherwise you'd get a syntax error.
Personally, I'd rewrite it a little further to make the behavior more explicit, but it's all up to you.
>>> [None if x is None else x*2 for x in v]
[4, 6, None, 8, None]

If you are able to use NaN instead of None, you can simplify your code, since NaN already behaves in the desired way, i.e. 2*nan evaluates to nan again, so you don't need to special-case it:
>>> nan = float('nan')
>>> v = [2, 3, nan, 4, nan]
>>> [2 * x for x in v]
[4, 6, nan, 8, nan]

Related

Python: from list group by elements after a specific trigger element [duplicate]

This question already has answers here:
Python3: How can I split a list based on condition?
(2 answers)
Closed last year.
I have a list like
a=['a',2,'[abcd]','bb',4,5,'kk','[efgh]',6,7,'no','[ijkl]',4,5,'lo']
So here we want group by after each '[]'
so the expected one would be
[['a',2],{'abcd': ['bb',4,5,'kk']},{'efgh': [6,7,'no']},{'ijkl': [4,5,'lo']}]
Any help would be appriciable
You can use groupby:
from itertools import groupby
a=['a',2,'[abcd]','bb',4,5,'kk','[efgh]',6,7,'no','[ijkl]',4,5,'lo']
def group_by_tag(li):
def tag_counter(x):
if isinstance(x, str) and x.startswith('[') and x.endswith(']'):
tag_counter.cnt += 1
return tag_counter.cnt
tag_counter.cnt = 0
return groupby(li, key=tag_counter)
Which you can use to make a list of tuples for each segment partitioned by the [tag]:
>>> x=[(k,list(l)) for k, l in group_by_tag(a)]
>>> x
[(0, ['a', 2]), (1, ['[abcd]', 'bb', 4, 5, 'kk']), (2, ['[efgh]', 6, 7, 'no']), (3, ['[ijkl]', 4, 5, 'lo'])]
And then create your desired mixed-type list from that:
>>> [v if k==0 else {v[0].strip('[]'):v[1:]} for k,v in x]
[['a', 2], {'abcd': ['bb', 4, 5, 'kk']}, {'efgh': [6, 7, 'no']}, {'ijkl': [4, 5, 'lo']}]
But consider that it is usually better to have a list of the same type of object to make processing that list easier.
If you want that, you could do:
>>> [{'no_tag':v} if k==0 else {v[0].strip('[]'):v[1:]} for k,v in x]
[{'no_tag': ['a', 2]}, {'abcd': ['bb', 4, 5, 'kk']}, {'efgh': [6, 7, 'no']}, {'ijkl': [4, 5, 'lo']}]
By "=>" I assume you want a dictionary if there is a preceding keyword and key be the word enclosed within the square brackets (if that's not what you intended feel free to comment and I'll edit this post)
import re
def sort(iterable):
result = {}
governing_match = None
for each in list(iterable):
match = re.findall(r"\[.{1,}\]", str(each))
if len(match) > 0:
governing_match = match[0][1:-1]
result[governing_match] = []
continue
result[governing_match].append(each)
return result
a=['[foo]', 'a' , 2 ,'[abcd]','bb',4,5,'kk','[efgh]',6,7,'no','[ijkl]',4,5,'lo']
for (k, v) in sort(a).items():
print(f"{k} : {v}")
Result :
foo : ['a', 2]
abcd : ['bb', 4, 5, 'kk']
efgh : [6, 7, 'no']
ijkl : [4, 5, 'lo']
The limitation of this is that every sequence should start with an element that is enclosed within square brackets.
You can use pairwise for that:
from itertools import pairwise
a=['a',2,'[abcd]','bb',4,5,'kk','[efgh]',6,7,'no','[ijkl]',4,5,'lo']
bracket_indices = [i for i, x in enumerate(a) if isinstance(x, str)
and x.startswith('[') and x.endswith(']')]
bracket_indices.append(len(a))
output = [a[:bracket_indices[0]]] # First item is special cased
output.extend({a[i][1:-1]: a[i+1:j]} for i, j in pairwise(bracket_indices))

How do I multiply elements in a list and find the sum in a simple way without numpy, zip etc

I have to multiply the same index of two lists and then find the sum of that.
Please help me out! Thank you.
Try this,
>>> A=[2, 3, -6, 7, 10, 11]
>>> B=[1, 2, 3, 4, 5, 6]
>>> sum([x * y for x, y in zip(A, B)])
134
Let me explain what I did in my answer, I used zip() python Built-in Function and this is what documentation mention about it.
Make an iterator that aggregates elements from each of the iterables.
Returns an iterator of tuples, where the i-th tuple contains the i-th element from each of the argument sequences or iterables. The iterator stops when the shortest input iterable is exhausted. With a single iterable argument, it returns an iterator of 1-tuples. With no arguments, it returns an empty iterator.
Little bit confusing, right? Check the below example:-
>>> A=[2, 3, -6, 7, 10, 11]
>>> B=[1, 2, 3, 4, 5, 6]
>>> zip(A,B)
<zip at 0x1fde73e6f88> # it display something like this (zip object)
>>> list(zip(A,B)) # for visualization purpose, convert zip object to list
[(2, 1), (3, 2), (-6, 3), (7, 4), (10, 5), (11, 6)]
I think you can get clear idea what happen inside zip() function. Then multiply each and every value in zip object using python List Comprehensions to answer your question more pythonic. So zip object now created for us a new value series using A and B list. We assigned those values for x and y, multiply them and save in list.
>>> [x * y for x, y in zip(A, B)]
[2, 6, -18, 28, 50, 66]
After all the steps we used sum() to calculate the Sum a list of numbers in Python.
>>> sum([2, 6, -18, 28, 50, 66])
134
That's all, if you didn't get anything please add a comment to this answer.
AB = [A[i] * B[i] for i in range(len(A))]
sum(AB)
Alternatively, try
AB = [value_ * B[i] for i, value_ in enumerate(A)]
sum(AB)
I believe you want this:
def lists(A,B):
C = 0
for i in range(len(A)):
C += (A[i] * B[i])
return C
Now you can call your method lists with lists A and B like this:
A=[2, 3, -6, 7, 10, 11]
B=[1, 2, 3, 4, 5, 6]
lists(A,B)
Which will return 134. Your code was wrong because of your indentation. You had put your return statement inside the for loop, so your code would return C value in the first iteration, which was 0 + 2*1.
with list comprehension:
A = [2, 3, -6, 7, 10, 11]
B = [1, 2, 3, 4, 5, 6]
print (sum([A[i]*B[i] for i in range(len(A))]))
output:
134
your code:
def lists(A, B):
C = 0
for i in range(len(A)):
C += (A[i] * B[i])
return C # <-----
A = [2, 3, -6, 7, 10, 11]
B = [1, 2, 3, 4, 5, 6]
print (lists(A,B))
NOTE: you need to put your return statement out of the for loop.
If return statement is reached during the execution of a function, it will exit from function, what does it mean in your case if you have return in your for loop, in first iteration return function will be reached and exit (you got result 2 because in first iteration yuo have 2*1)

Difference between normal for loop statement and for loop statement inside a list initialization in python [duplicate]

I have the following code:
[x ** 2 for x in range(10)]
When I run it in the Python shell, it returns:
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
I've searched and it seems this is called a list comprehension and similarly there seem to be set/dict comprehensions and generator expressions. But how does it work?
From the documentation:
List comprehensions provide a concise way to create lists. Common applications are to make new lists where each element is the result of some operations applied to each member of another sequence or iterable, or to create a subsequence of those elements that satisfy a certain condition.
About your question, the list comprehension does the same thing as the following "plain" Python code:
>>> l = []
>>> for x in range(10):
... l.append(x**2)
>>> l
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
How do you write it in one line? Hmm...we can...probably...use map() with lambda:
>>> list(map(lambda x: x**2, range(10)))
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
But isn't it clearer and simpler to just use a list comprehension?
>>> [x**2 for x in range(10)]
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Basically, we can do anything with x. Not only x**2. For example, run a method of x:
>>> [x.strip() for x in ('foo\n', 'bar\n', 'baz\n')]
['foo', 'bar', 'baz']
Or use x as another function's argument:
>>> [int(x) for x in ('1', '2', '3')]
[1, 2, 3]
We can also, for example, use x as the key of a dict object. Let's see:
>>> d = {'foo': '10', 'bar': '20', 'baz': '30'}
>>> [d[x] for x in ['foo', 'baz']]
['10', '30']
How about a combination?
>>> d = {'foo': '10', 'bar': '20', 'baz': '30'}
>>> [int(d[x].rstrip('0')) for x in ['foo', 'baz']]
[1, 3]
And so on.
You can also use if or if...else in a list comprehension. For example, you only want odd numbers in range(10). You can do:
>>> l = []
>>> for x in range(10):
... if x%2:
... l.append(x)
>>> l
[1, 3, 5, 7, 9]
Ah that's too complex. What about the following version?
>>> [x for x in range(10) if x%2]
[1, 3, 5, 7, 9]
To use an if...else ternary expression, you need put the if ... else ... after x, not after range(10):
>>> [i if i%2 != 0 else None for i in range(10)]
[None, 1, None, 3, None, 5, None, 7, None, 9]
Have you heard about nested list comprehension? You can put two or more fors in one list comprehension. For example:
>>> [i for x in [[1, 2, 3], [4, 5, 6]] for i in x]
[1, 2, 3, 4, 5, 6]
>>> [j for x in [[[1, 2], [3]], [[4, 5], [6]]] for i in x for j in i]
[1, 2, 3, 4, 5, 6]
Let's talk about the first part, for x in [[1, 2, 3], [4, 5, 6]] which gives [1, 2, 3] and [4, 5, 6]. Then, for i in x gives 1, 2, 3 and 4, 5, 6.
Warning: You always need put for x in [[1, 2, 3], [4, 5, 6]] before for i in x:
>>> [j for j in x for x in [[1, 2, 3], [4, 5, 6]]]
Traceback (most recent call last):
File "<input>", line 1, in <module>
NameError: name 'x' is not defined
We also have set comprehensions, dict comprehensions, and generator expressions.
set comprehensions and list comprehensions are basically the same, but the former returns a set instead of a list:
>>> {x for x in [1, 1, 2, 3, 3, 1]}
{1, 2, 3}
It's the same as:
>>> set([i for i in [1, 1, 2, 3, 3, 1]])
{1, 2, 3}
A dict comprehension looks like a set comprehension, but it uses {key: value for key, value in ...} or {i: i for i in ...} instead of {i for i in ...}.
For example:
>>> {i: i**2 for i in range(5)}
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
And it equals:
>>> d = {}
>>> for i in range(5):
... d[i] = i**2
>>> d
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
Does (i for i in range(5)) give a tuple? No!, it's a generator expression. Which returns a generator:
>>> (i for i in range(5))
<generator object <genexpr> at 0x7f52703fbca8>
It's the same as:
>>> def gen():
... for i in range(5):
... yield i
>>> gen()
<generator object gen at 0x7f5270380db0>
And you can use it as a generator:
>>> gen = (i for i in range(5))
>>> next(gen)
0
>>> next(gen)
1
>>> list(gen)
[2, 3, 4]
>>> next(gen)
Traceback (most recent call last):
File "<input>", line 1, in <module>
StopIteration
Note: If you use a list comprehension inside a function, you don't need the [] if that function could loop over a generator. For example, sum():
>>> sum(i**2 for i in range(5))
30
Related (about generators): Understanding Generators in Python.
There are list, dictionary, and set comprehensions, but no tuple comprehensions (though do explore "generator expressions").
They address the problem that traditional loops in Python are statements (don't return anything) not expressions which return a value.
They are not the solution to every problem and can be rewritten as traditional loops. They become awkward when state needs to be maintained & updated between iterations.
They typically consist of:
[<output expr> <loop expr <input expr>> <optional predicate expr>]
but can be twisted in lots of interesting and bizarre ways.
They can be analogous to the traditional map() and filter() operations which still exist in Python and continue to be used.
When done well, they have a high satisfaction quotient.
If you prefer a more visual way of figuring out what's going on then maybe this will help:
# for the example in the question...
y = []
for x in range(10):
y += [x**2]
# is equivalent to...
y = [x**2 for x in range(10)]
# for a slightly more complex example, it is useful
# to visualize where the various x's end up...
a = [1,2,3,4]
b = [3,4,5,6]
c = []
for x in a:
if x in b:
c += [x]
# \ \ /
# \ _____\______/
# \ / \
# \/ \
# /\ \
# / \ \
# / \ \
c = [x for x in a if x in b]
print(c)
...produces the output [3, 4]
I've seen a lot of confusion lately (on other SO questions and from coworkers) about how list comprehensions work. A wee bit of math education can help with why the syntax is like this, and what list comprehensions really mean.
The syntax
It's best to think of list comprehensions as predicates over a set/collection, like we would in mathematics by using set builder notation. The notation actually feels pretty natural to me, because I hold an undergrad degree in Mathematics. But forget about me, Guido van Rossum (inventor of Python) holds a masters in Mathematics and has a math background.
Set builder notation crash course
Here's the (very basics) of how set builder notation works:
So, this set builder notation represents the set of numbers that are strictly positive (i.e. [1,2,3,4,...]).
Points of confusion
1) The predicate filter in set builder notation only specifies which items we want to keep, and list comprehension predicates do the same thing. You don't have to include special logic for omitting items, they are omitted unless included by the predicate. The empty predicate (i.e. no conditional at the end) includes all items in the given collection.
2) The predicate filter in set builder notation goes at the end, and similarly in list comprehensions. (some) Beginners think something like [x < 5 for x in range(10)] will give them the list [0,1,2,3,4], when in fact it outputs [True, True, True, True, True, False, False, False, False, False]. We get the output [True, True, True, True, True, False, False, False, False, False] because we asked Python to evaluate x < 5 for all items in range(10). No predicate implies that we get everything from the set (just like in set builder notation).
If you keep set builder notation in the back of your mind while using list comprehensions, they're a bit easier to swallow.
HTH!
Introduction
A list comprehension is a high level, declarative way to create a list in Python. The main benefits of comprehensions are readability and maintainability. A lot of people find them very readable, and even developers who have never seen them before can usually guess correctly what it means.
# Snippet 1
squares = [n ** 2 for n in range(5)]
# Snippet 2
squares = []
for n in range(5):
squares.append(n ** 2)
Both snippets of code will produce squares to be equal to [0, 1, 4, 9, 16].
Notice that in the first snippet, what you type is declaring what kind of list you want, while the second is specifying how to create it. This is why a comprehension is a high-level and declarative.
Syntax
[EXPRESSION for VARIABLE in SEQUENCE]
EXPRESSION is any Python expression, but it is typical to have some variable in it. This variable is stated in VARIABLE field. SEQUENCE defines the source of values the variable enumerates through.
Considering Snippet 1, [n ** 2 for n in range(5)]:
EXPRESSION is n ** 2
VARIABLE is n
SEQUENCE is range(5)
Notice that if you check the type of squares you will get that the list comprehension is just a regular list:
>>> type(squares)
<class 'list'>
More about EXPRESSION
The expression can be anything that reduces to a value:
Arithmetic expressions such as n ** 2 + 3 * n + 1
A function call like f(n) using n as variable
A slice operation like s[::-1]
Method calls bar.foo()
...
Some examples:
>>> [2 * x + 3 for x in range(5)]
[3, 5, 7, 9, 11]
>>> [abs(num) for num in range(-5, 5)]
[5, 4, 3, 2, 1, 0, 1, 2, 3, 4]
>>> animals = ['dog', 'cat', 'lion', 'tiger']
>>> [animal.upper() for animal in animals]
['DOG', 'CAT', 'LION', 'TIGER']
Filtering:
The order of elements in the final list is determined by the order of SEQUENCE. However, you can filter out elements adding an if clause:
[EXPRESSION for VARIABLE in SEQUENCE if CONDITION]
CONDITION is an expression that evaluates to True or False. Technically, the condition doesn't have to depend upon VARIABLE, but it typically uses it.
Examples:
>>> [n ** 2 for n in range(5) if n % 2 == 0]
[0, 4, 16]
>>> animals = ['dog', 'cat', 'lion', 'tiger']
>>> [animal for animal in animals if len(animal) == 3]
['dog', 'cat']
Also, remember that Python allows you to write other kinds of comprehensions other than lists:
dictionary comprehensions
set comprehensions

sort() in python does not completely work as expected

If l is a list, say [9,3,4,1,6], I know that l.sort() will sort the list in place. I also know that m=sorted(l) will create a new list, m, as a sorted version of l.
What I don't understand is why m=l.sort(), does not create m as a sorted version of l, since this appears to be an assignment of a sorted list to m. The problem can be seen from the following
In [37]: l = [9,3,4,1,6]
In [38]: m=l.sort()
In [39]: m
In [40]: type(m)
Out[40]: NoneType
Of course if I do the following, m becomes a sorted version of l
In [41]: l = [9,3,4,1,6]
In [42]: l.sort()
In [43]: m=l
In [44]: m
Out[44]: [1, 3, 4, 6, 9]
Can someone explain why m=l.sort() doesn't work as an assignment?
list.sort mutates your list to sort it, it does not return a new one.
l = [9, 3, 4, 1, 6]
l.sort()
l # [1, 3, 4, 6, 9]
If you want to create a new sorted list, use sorted.
l = [9, 3, 4, 1, 6]
m = sorted(l)
l # [9, 3, 4, 1, 6]
m # [1, 3, 4, 6, 9]
l.sort() has no return value (see here), so when you do
m = l.sort()
you're assigning m to nothing
sort() works directly on the list and has a None return type.
So, if you want to assign it to some variable and keep the original list, you can do it the following way:
l = [9,3,4,1,6]
m = sorted(l)
print(m)
Output:
[1, 3, 4, 6, 9]
It will keep the original list as it is.

How do I find x given y or vice versa in numpy?

For example, I have the following arrays:
x = [0, 1, 2, 3, 4.5, 5]
y = [2, 8, 3, 7, 8, 1]
I would like to be able to do the following given x:
>>> what_is_y_when_x_is(2)
(2, 3)
>>> what_is_y_when_x_is(3.1) # Perhaps set rules to round to nearest (or up or down)
(3, 7)
On the other hand, when given y:
>>> what_is_x_when_y_is(2)
(0, 2)
>>> what_is_x_when_y_is(max(y))
([1, 4.5], 8)
The circumstances of this problem
I could have plotted y versus x using a closed analytical function, which should be very easy by just calling foo_function(x). However, I'm running numerical simulations whose data plots do not have closed analytical solutions.
Attempted solution
I've tackled similar problems before and approached them roughly this way:
what_is_y_when_x_is(some_x)
Search the array x for some_x.
Get its index, i.
Pick up y[i].
Question
Is there a better way to do this? Perhaps a built-in numpy function or a better algorithm?
You should look at numpy.searchsorted and also numpy.interp. Both of those look like they might do the trick. Here is an example:
import numpy as np
x = np.array([0, 1, 2, 3, 4.5, 5])
y = np.array([2, 8, 3, 7, 8, 1])
# y should be sorted for both of these methods
order = y.argsort()
y = y[order]
x = x[order]
def what_is_x_when_y_is(input, x, y):
return x[y.searchsorted(input, 'left')]
def interp_x_from_y(input, x, y):
return np.interp(input, y, x)
print what_is_x_when_y_is(7, x, y)
# 3
print interp_x_from_y(1.5, x, y)
# 2.5
You could use the bisect module for this. This is pure python - no numpy here:
>>> x = [0, 1, 2, 3, 4.5, 5]
>>> y = [2, 8, 3, 7, 8, 1]
>>> x_lookup = sorted(zip(x, y))
>>> y_lookup = sorted(map(tuple, map(reversed, zip(x, y))))
>>>
>>> import bisect
>>> def pair_from_x(x):
... return x_lookup[min(bisect.bisect_left(x_lookup, (x,)), len(x_lookup)-1)]
...
>>> def pair_from_y(y):
... return tuple(reversed(y_lookup[min(bisect.bisect_left(y_lookup, (y,)), len(y_lookup)-1)]))
...
And some examples of using it:
>>> pair_from_x(0)
(0, 2)
>>> pair_from_x(-2)
(0, 2)
>>> pair_from_x(2)
(2, 3)
>>> pair_from_x(3)
(3, 7)
>>> pair_from_x(7)
(5, 1)
>>>
>>> pair_from_y(0)
(5, 1)
>>> pair_from_y(1)
(5, 1)
>>> pair_from_y(3)
(2, 3)
>>> pair_from_y(4)
(3, 7)
>>> pair_from_y(8)
(1, 8)
The way you described is, as far as I'm considered, a good way. I'm not sure if you are, but I think you could use the .index(...) method on your array:
>>> li
['I', 'hope', 'this', 'answer', 'helps', 'you']
>>> li.index("hope")
1
Other than that, you might want to consider one array op "Points" which have an x and a y, though I'm not sure if this is possible of course. That way you won't have to keep two arrays in sync (same number of elements).
I don't see any problem with your pipeline. You can write a snippet base on numpy.where to implement it efficiently. Note that you will have to pass your lists as numpy arrays first (this can be included in the function).
Below is an example of a function doing the job, with an option to round the target (I have included a which array argument, so everything can be done in just one function, whatever you want to search in x or y). Note that one of the output will be a numpy array, so modify it to convert it to anything you want (liste, tuple, etc ...).
import numpy as np
def pick(x_array, y_array, target, which_array='x', round=True):
# ensure that x and y are numpy arrays
x_array, y_array = np.array(x_array), np.array(y_array)
# optional: round to the nearest. True by default
if round==True:
target = np.round(target)
if which_array == 'x': # look for the target in x_array
return target, y_array[np.where(x_array == target)[0]]
if which_array == 'y': # look for the target in y_array
return x_array[np.where(y_array == target)[0]], target
Results given by your examples:
# >>> what_is_y_when_x_is(2)
pick(x, y, 2, 'x')
(2, array([3]))
# >>> what_is_y_when_x_is(3.1)
pick(x, y, 3.1, 'x')
3.0, array([7]))
# >>> what_is_y_when_x_is(2)
pick(x, y, 2, 'y')
(array([ 0.]), 2)
# >>> what_is_x_when_y_is(max(y))
pick(x, y, max(y), 'y')
(array([ 1. , 4.5]), 8)
Here is a revised version of code provided by # Bi Rico:
import numpy as np
x = np.array([0, 1, 2, 3, 4.5, 5])
y = np.array([2, 8, 3, 7, 8, 1])
# y should be sorted for both of these methods
order = np.argsort(y)
y = y[order]
x = x[order]
def what_is_x_when_y_is(input, x, y):
return x[y.searchsorted(input, 'left')]
print(what_is_x_when_y_is(7, x, y))
# 3
This worked for me:
def what_is_y_when_x_is(value, x, y, tolerance=1e-3):
return [(xi, yi) for (xi, yi) in zip(x, y) if abs(xi - value) <= tolerance]
Notice that rather than comparing for equality the code above performs a "close enough" equality test. The default tolerance is set up to 0.001 (you can use any other value). Here are some examples of use:
>>> x = [0, 1, 2, 3, 4.5, 5]
>>> y = [2, 8, 3, 7, 8, 1]
>>> what_is_y_when_x_is(0, x, y)
[(0, 2)]
>>> what_is_y_when_x_is(1, x, y, tolerance=.1)
[(1, 8)]
>>> what_is_y_when_x_is(2, x, y, tolerance=1)
[(1, 8), (2, 3), (3, 7)]
>>> what_is_y_when_x_is(4, x, y, tolerance=.5)
[(4.5, 8)]

Resources