Numpy: Selecting Rows based on Multiple Conditions on Some of its Elements - python-3.x

I believe this is not a duplicate question, although there are questions that are fairly close to this one on the website. I would like to isolate a row from a numpy list given a set of conditions for some of its elements. Here is an example, consider the array Z:
>>> Z = [[1,0,3,4], [1,1,3,6], [1,2,3,9], [1,3,4,0], [2,1,4,5]]
>>> Z = np.array(Z)
>>> Z
array([[1, 0, 3, 4],
[1, 1, 3, 6],
[1, 2, 3, 9],
[1, 3, 4, 0],
[2, 1, 4, 5]])
and say I would like to isolate the row whose first and second element are both 1. The command that executes that should output the row
np.array([[1, 1, 3, 6]])
However, if I follow this popular question, and make an intuitive extension, such as:
Z[Z[:,0] == 1 & Z[:,1] == 1, :]
I get:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Is there any quick fix to that? I do not want to iterate over my list. I was wondering if there is a quick "numpy" way for it.

Elegant is np.equal
Z[np.equal(Z[:, [0,1]], 1).all(axis=1)]
Or:
Z[np.equal(Z[:,0], 1) & np.equal(Z[:,1], 1)]

More simple
print (Z[(Z[:,0]==1)&(Z[:,1]==1)])
or
print (Z[(Z[:,0]==1)&(Z[:,1]==1),:])
You got
[[1 1 3 6]]

Related

How can I get multiple output variables into a list?

I'm wondering if there's a way of getting multiple outputs from a function into a list. I'm not interested in creating a list inside of a function for reasons I'm not going to waste your time going into.
I know how many output variables I am expecting, but only through using the annotations["return"] expression (or whatever you call that, sorry for the noobish terminology) and this changes from case to case, which is why I need this to be dynamic.
I know I can use lists as multiple variables using function(*myList), but I'm interested in if there's a way of doing the equivalent when receiving return values from a function.
Cheers!
Pseudocode:
function():
x = 1
y = 2
return x, y
variables = function()
print(variables[0], " and ", variables[1]
result should be = "1 and 2"
yes, with the unpacking assignments expression ex a,b,c= myfunction(...), you can put * in one of those to make it take a variable number of arguments
>>> a,b,c=range(3) #if you know that the thing contains exactly 3 elements you can do this
>>> a,b,c
(0, 1, 2)
>>> a,b,*c=range(10) #for when you know that there at least 2 or more the first 2 will be in a and b, and whatever else in c which will be a list
>>> a,b,c
(0, 1, [2, 3, 4, 5, 6, 7, 8, 9])
>>> a,*b,c=range(10)
>>> a,b,c
(0, [1, 2, 3, 4, 5, 6, 7, 8], 9)
>>> *a,b,c=range(10)
>>> a,b,c
([0, 1, 2, 3, 4, 5, 6, 7], 8, 9)
>>>
additionally you can return from a function whatever you want, a list, a tuple, a dict, etc, but only one thing
>>> def fun():
return 1,"boo",[1,2,3],{1:10,3:23}
>>> fun()
(1, 'boo', [1, 2, 3], {1: 10, 3: 23})
>>>
in this example it return a tuple with all that stuff because , is the tuple constructor, so it make a tuple first (your one thing) and return it

Python: How to reset the list to original value inside the for loop for every iteration?

I'm trying to pop every values one by one from a list in order to find the min and max sum. While doing that I'm resetting the list to its original value after every iteration, but it doesn't seems to be working...
a=[1,2,3,4,5]
res=[]
for i in range(len(a)):
#print(a)
lst=a
#print(lst)
lst.pop(i)
print(lst)
res.append(sum(lst))
print(min(res))
print(max(res))
[2, 3, 4, 5]
[2, 4, 5]
[2, 4]
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-33-42daa1037d37> in <module>
5 lst=a
6 #print(lst)
----> 7 lst.pop(i)
8 print(lst)
9 res.append(sum(lst))
IndexError: pop index out of range
I'm resetting the "lst" to "a" after every iteration, but its not working as expected.
Expected result:
[2, 3, 4, 5]
[1, 3, 4, 5]
[1, 2, 4, 5]
[1, 2, 3, 5]
[1, 2, 3, 4]
10
14
Any help would be appreciated!
The operator "=" doesn't duplicate your list into two different object.
In fact "lst" and "a" both refer to the same object.
Which means if you modify "lst" you will also modify "a" :
>>> a=[1,2,3,4,5]
>>> lst = a
>>> lst.pop(0)
>>> print(a)
[2, 3, 4, 5]
You can change this behavior using the module copy and its function deepcopy. It will duplicate your list and not affect your original one.
import copy
a=[1,2,3,4,5]
res=[]
for i in range(len(a)):
#print(a)
lst= copy.deepcopy(a)
#print(lst)
lst.pop(i)
print(lst)
res.append(sum(lst))
print(min(res))
print(max(res))

Python: Can't change a dictionary field while iterating over it

I have this list:
>>> stud
[19, 11, 6, 26]
If I loop over the 'draws' field, I get the following:
>>> [aluDict[str(s)]['draws'] for s in stud]
[1, 0, 1, 0]
Now I want to add 1 to each of those, so as to obtain (potentially) the following:
>>> [aluDict[str(s)]['draws'] for s in stud]
[2, 1, 2, 1]
In order to do so, I tried this:
>>> [aluDict[str(s)]['draws']+=1 for s in stud]
File "<stdin>", line 1
[aluDict[str(s)]['draws']+=1 for s in stud]
^
SyntaxError: invalid syntax
If I do it individually, I can accomplish that:
>>> aluDict[str(19)]['draws']
2
>>> aluDict[str(19)]['draws']+=1
>>> aluDict[str(19)]['draws']
3
Why is this happening? What am I missing?
Thanks!
If you just want the list [2, 1, 2, 1] and don't want to actually change any of the values stored in aluDict, use +1:
[aluDict[str(s)]['draws']+1 for s in stud]

Difference between normal for loop statement and for loop statement inside a list initialization in python [duplicate]

I have the following code:
[x ** 2 for x in range(10)]
When I run it in the Python shell, it returns:
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
I've searched and it seems this is called a list comprehension and similarly there seem to be set/dict comprehensions and generator expressions. But how does it work?
From the documentation:
List comprehensions provide a concise way to create lists. Common applications are to make new lists where each element is the result of some operations applied to each member of another sequence or iterable, or to create a subsequence of those elements that satisfy a certain condition.
About your question, the list comprehension does the same thing as the following "plain" Python code:
>>> l = []
>>> for x in range(10):
... l.append(x**2)
>>> l
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
How do you write it in one line? Hmm...we can...probably...use map() with lambda:
>>> list(map(lambda x: x**2, range(10)))
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
But isn't it clearer and simpler to just use a list comprehension?
>>> [x**2 for x in range(10)]
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Basically, we can do anything with x. Not only x**2. For example, run a method of x:
>>> [x.strip() for x in ('foo\n', 'bar\n', 'baz\n')]
['foo', 'bar', 'baz']
Or use x as another function's argument:
>>> [int(x) for x in ('1', '2', '3')]
[1, 2, 3]
We can also, for example, use x as the key of a dict object. Let's see:
>>> d = {'foo': '10', 'bar': '20', 'baz': '30'}
>>> [d[x] for x in ['foo', 'baz']]
['10', '30']
How about a combination?
>>> d = {'foo': '10', 'bar': '20', 'baz': '30'}
>>> [int(d[x].rstrip('0')) for x in ['foo', 'baz']]
[1, 3]
And so on.
You can also use if or if...else in a list comprehension. For example, you only want odd numbers in range(10). You can do:
>>> l = []
>>> for x in range(10):
... if x%2:
... l.append(x)
>>> l
[1, 3, 5, 7, 9]
Ah that's too complex. What about the following version?
>>> [x for x in range(10) if x%2]
[1, 3, 5, 7, 9]
To use an if...else ternary expression, you need put the if ... else ... after x, not after range(10):
>>> [i if i%2 != 0 else None for i in range(10)]
[None, 1, None, 3, None, 5, None, 7, None, 9]
Have you heard about nested list comprehension? You can put two or more fors in one list comprehension. For example:
>>> [i for x in [[1, 2, 3], [4, 5, 6]] for i in x]
[1, 2, 3, 4, 5, 6]
>>> [j for x in [[[1, 2], [3]], [[4, 5], [6]]] for i in x for j in i]
[1, 2, 3, 4, 5, 6]
Let's talk about the first part, for x in [[1, 2, 3], [4, 5, 6]] which gives [1, 2, 3] and [4, 5, 6]. Then, for i in x gives 1, 2, 3 and 4, 5, 6.
Warning: You always need put for x in [[1, 2, 3], [4, 5, 6]] before for i in x:
>>> [j for j in x for x in [[1, 2, 3], [4, 5, 6]]]
Traceback (most recent call last):
File "<input>", line 1, in <module>
NameError: name 'x' is not defined
We also have set comprehensions, dict comprehensions, and generator expressions.
set comprehensions and list comprehensions are basically the same, but the former returns a set instead of a list:
>>> {x for x in [1, 1, 2, 3, 3, 1]}
{1, 2, 3}
It's the same as:
>>> set([i for i in [1, 1, 2, 3, 3, 1]])
{1, 2, 3}
A dict comprehension looks like a set comprehension, but it uses {key: value for key, value in ...} or {i: i for i in ...} instead of {i for i in ...}.
For example:
>>> {i: i**2 for i in range(5)}
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
And it equals:
>>> d = {}
>>> for i in range(5):
... d[i] = i**2
>>> d
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
Does (i for i in range(5)) give a tuple? No!, it's a generator expression. Which returns a generator:
>>> (i for i in range(5))
<generator object <genexpr> at 0x7f52703fbca8>
It's the same as:
>>> def gen():
... for i in range(5):
... yield i
>>> gen()
<generator object gen at 0x7f5270380db0>
And you can use it as a generator:
>>> gen = (i for i in range(5))
>>> next(gen)
0
>>> next(gen)
1
>>> list(gen)
[2, 3, 4]
>>> next(gen)
Traceback (most recent call last):
File "<input>", line 1, in <module>
StopIteration
Note: If you use a list comprehension inside a function, you don't need the [] if that function could loop over a generator. For example, sum():
>>> sum(i**2 for i in range(5))
30
Related (about generators): Understanding Generators in Python.
There are list, dictionary, and set comprehensions, but no tuple comprehensions (though do explore "generator expressions").
They address the problem that traditional loops in Python are statements (don't return anything) not expressions which return a value.
They are not the solution to every problem and can be rewritten as traditional loops. They become awkward when state needs to be maintained & updated between iterations.
They typically consist of:
[<output expr> <loop expr <input expr>> <optional predicate expr>]
but can be twisted in lots of interesting and bizarre ways.
They can be analogous to the traditional map() and filter() operations which still exist in Python and continue to be used.
When done well, they have a high satisfaction quotient.
If you prefer a more visual way of figuring out what's going on then maybe this will help:
# for the example in the question...
y = []
for x in range(10):
y += [x**2]
# is equivalent to...
y = [x**2 for x in range(10)]
# for a slightly more complex example, it is useful
# to visualize where the various x's end up...
a = [1,2,3,4]
b = [3,4,5,6]
c = []
for x in a:
if x in b:
c += [x]
# \ \ /
# \ _____\______/
# \ / \
# \/ \
# /\ \
# / \ \
# / \ \
c = [x for x in a if x in b]
print(c)
...produces the output [3, 4]
I've seen a lot of confusion lately (on other SO questions and from coworkers) about how list comprehensions work. A wee bit of math education can help with why the syntax is like this, and what list comprehensions really mean.
The syntax
It's best to think of list comprehensions as predicates over a set/collection, like we would in mathematics by using set builder notation. The notation actually feels pretty natural to me, because I hold an undergrad degree in Mathematics. But forget about me, Guido van Rossum (inventor of Python) holds a masters in Mathematics and has a math background.
Set builder notation crash course
Here's the (very basics) of how set builder notation works:
So, this set builder notation represents the set of numbers that are strictly positive (i.e. [1,2,3,4,...]).
Points of confusion
1) The predicate filter in set builder notation only specifies which items we want to keep, and list comprehension predicates do the same thing. You don't have to include special logic for omitting items, they are omitted unless included by the predicate. The empty predicate (i.e. no conditional at the end) includes all items in the given collection.
2) The predicate filter in set builder notation goes at the end, and similarly in list comprehensions. (some) Beginners think something like [x < 5 for x in range(10)] will give them the list [0,1,2,3,4], when in fact it outputs [True, True, True, True, True, False, False, False, False, False]. We get the output [True, True, True, True, True, False, False, False, False, False] because we asked Python to evaluate x < 5 for all items in range(10). No predicate implies that we get everything from the set (just like in set builder notation).
If you keep set builder notation in the back of your mind while using list comprehensions, they're a bit easier to swallow.
HTH!
Introduction
A list comprehension is a high level, declarative way to create a list in Python. The main benefits of comprehensions are readability and maintainability. A lot of people find them very readable, and even developers who have never seen them before can usually guess correctly what it means.
# Snippet 1
squares = [n ** 2 for n in range(5)]
# Snippet 2
squares = []
for n in range(5):
squares.append(n ** 2)
Both snippets of code will produce squares to be equal to [0, 1, 4, 9, 16].
Notice that in the first snippet, what you type is declaring what kind of list you want, while the second is specifying how to create it. This is why a comprehension is a high-level and declarative.
Syntax
[EXPRESSION for VARIABLE in SEQUENCE]
EXPRESSION is any Python expression, but it is typical to have some variable in it. This variable is stated in VARIABLE field. SEQUENCE defines the source of values the variable enumerates through.
Considering Snippet 1, [n ** 2 for n in range(5)]:
EXPRESSION is n ** 2
VARIABLE is n
SEQUENCE is range(5)
Notice that if you check the type of squares you will get that the list comprehension is just a regular list:
>>> type(squares)
<class 'list'>
More about EXPRESSION
The expression can be anything that reduces to a value:
Arithmetic expressions such as n ** 2 + 3 * n + 1
A function call like f(n) using n as variable
A slice operation like s[::-1]
Method calls bar.foo()
...
Some examples:
>>> [2 * x + 3 for x in range(5)]
[3, 5, 7, 9, 11]
>>> [abs(num) for num in range(-5, 5)]
[5, 4, 3, 2, 1, 0, 1, 2, 3, 4]
>>> animals = ['dog', 'cat', 'lion', 'tiger']
>>> [animal.upper() for animal in animals]
['DOG', 'CAT', 'LION', 'TIGER']
Filtering:
The order of elements in the final list is determined by the order of SEQUENCE. However, you can filter out elements adding an if clause:
[EXPRESSION for VARIABLE in SEQUENCE if CONDITION]
CONDITION is an expression that evaluates to True or False. Technically, the condition doesn't have to depend upon VARIABLE, but it typically uses it.
Examples:
>>> [n ** 2 for n in range(5) if n % 2 == 0]
[0, 4, 16]
>>> animals = ['dog', 'cat', 'lion', 'tiger']
>>> [animal for animal in animals if len(animal) == 3]
['dog', 'cat']
Also, remember that Python allows you to write other kinds of comprehensions other than lists:
dictionary comprehensions
set comprehensions

What is the pythonic way of extracting (& removing) a slice from a list? Or... What is the "inverse" of the slice+=list operator - python

I know how insert a list into a list, "slice+=list" ...
master=[0,1,2,3,7,8,9]
master[:4]+=[4,5,6] # insert 4,5,6
(crudely) The inverse of this operation is removing a slice 4:7 from the list, I tried:
extracted=del master[4:7]
But this gives a syntax error "SyntaxError: invalid syntax".
Likewise the inverse slice operator "-=" doesn't appear to exist.
As a workaround I have used the following:
extracted=master[4:7]; del master[4:7]
This "works" and the "extracted" is the subslice removed from "master", e.g.
print dict(master=master,extracted=extracted)
Output:
{'extracted': [4, 5, 6], 'master': [0, 1, 2, 3, 7, 8, 9]}
Is there a better/pythonic/simpler way ?
In particular I don't like the repeated [4:7] in:
extracted=master[4:7]; del master[4:7]"
Because of potential side-effects: eg
extracted=master[randint(0,3):randint(7,10)]; del master[randint(0,3):randint(7,10)]
i.e. the following reads much better, and would have no "side-effects"...
extracted=del master[randint(0,3):randint(7,10)]
Any hints? Is there a slice "-=" operator I could have used to invert the action of the slice "+=" operator?
Your best bet is to use a slice:
-> s = slice(4, 7)
-> master
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
--> extracted = master[s]
--> extracted
[4, 5, 6]
--> del master[s]
--> master
[0, 1, 2, 3, 7, 8, 9]
It still requires two commands, but you can use a single object to respresent the piece you want.
For me the cleaner option is as follows:
>>> L=[1,2,3,4,5,6] #Define the list
>>> L #Check it
[1, 2, 3, 4, 5, 6]
>>> extract= L[3:6] ; L[3:6]=[] #Assign the slice to 'extract'; delete the slice
>>> L #Items removed from the list
[1, 2, 3]
>>> extract #And assigned to extract
[4, 5, 6]
Cheers!

Resources