Given a Python function definition, is there some tool that can check that all variables used inside the function are local (either passed in as a parameter or declared within the function)?
To get all the local variables you can use dir():
e.g.
def func(a,b):
name='John'
city='New York'
age=33
#to print all the local variables inside this function.
print(dir())
func(2,3)
#output
['a', 'age', 'b', 'city', 'name']
You can also use locals():
def func(a,b):
name='John'
city='New York'
age=33
#to print all the local variables inside this function.
print(locals())
func(2,3)
#output
{'a': 2, 'b': 3, 'name': 'John', 'city': 'New York', 'age': 33}
The indentation is best feature in python so when the function , loop or any other is used in python we get single tab as indent from left so we can know about local and global variables in python
a=10
def func(a,b):
name='John'
city='New York'
age=33
#to print all the local variables inside this function.
print(dir())
func(2,3)
#output
['a', 'age', 'b', 'city', 'name']
In the above code the a=10 is global and the a inside the function parenthesis is local.
Related
I have a program which producing and modifying a list of "n" elements/members, n remaining constant throughout a particular run of the program. (The value of "n" might change in the next run).
Each member in the list is a "sub-list"! Each of these sub-list elements are not only of variable lengths, but are also dynamic and might keep changing while the program keeps running.
So, eventually, at some given point, my list would look something like (assuming n=3):
[['1', '2'], ['a', 'b', 'c', 'd'], ['x', 'y', 'z']]
I want the output to be like the following:
['1ax', '1ay', '1az', '1bx', '1by', '1bz',
'1cx', '1cy', '1cz', '1dx', '1dy', '1dz',
'2ax', '2ay', '2az', '2bx', '2by', '2bz',
'2cx', '2cy', '2cz', '2dx', '2dy', '2dz']
i.e. a list with exactly (2 * 3 * 4) elements where each element is of length exactly 3 and has exactly 1 member from each of the "sub-lists".
Easiest is itertools.product:
from itertools import product
lst = [['1', '2'], ['a', 'b', 'c', 'd'], ['x', 'y', 'z']]
output = [''.join(p) for p in product(*lst)]
# OR
output = list(map(''.join, product(*lst)))
# ['1ax', '1ay', '1az', '1bx', '1by', '1bz',
# '1cx', '1cy', '1cz', '1dx', '1dy', '1dz',
# '2ax', '2ay', '2az', '2bx', '2by', '2bz',
# '2cx', '2cy', '2cz', '2dx', '2dy', '2dz']
A manual implementation specific to strings could look like this:
def prod(*pools):
if pools:
*rest, pool = pools
for p in prod(*rest):
for el in pool:
yield p + el
else:
yield ""
list(prod(*lst))
# ['1ax', '1ay', '1az', '1bx', '1by', '1bz',
# '1cx', '1cy', '1cz', '1dx', '1dy', '1dz',
# '2ax', '2ay', '2az', '2bx', '2by', '2bz',
# '2cx', '2cy', '2cz', '2dx', '2dy', '2dz']
Code 1: for loop
def foo():
one = '1'
two = '2'
three = '3'
d = {}
for name in ('one', 'two', 'three'):
d[name] = eval(name)
print(d)
foo()
output:
{'one': '1', 'two': '2', 'three': '3'}
Code 2: dict comprehension
def foo():
one = '1'
two = '2'
three = '3'
print({name: eval(name) for name in ('one', 'two', 'three')})
foo()
output:
NameError: name 'one' is not defined
Code 3: add global keyword
def foo():
global one, two, three # why?
one = '1'
two = '2'
three = '3'
print({name: eval(name) for name in ('one', 'two', 'three')})
foo()
output:
{'one': '1', 'two': '2', 'three': '3'}
Dict comprehensions and generator comprehensions create their own local scope. According to the definition of the closure (or not the closure here), but why can't Code 2 access the variable one[,two,three] of the outer function foo? However, Code 3 can successfully create a dictionary by setting the variable one[,two,three] to global?
So is it because the eval function and the dict comprehensions have different scopes?
Hope someone help me, I will be grateful!
To understand whats happening, try this:
def foo():
global one
one = '1'
two = '2'
print({'locals, global': (locals(), globals()) for _ in range(1)})
foo()
Output
{'locals, global': ({'_': 0, '.0': <range_iterator object at ...>},
{'__name__': '__main__', '__package__': None, ..., 'one': '1'})}
The builtin eval(expression) is a shortcut for eval(expression[, globals[, locals]]).
As you see in the previous output, locals() is not local symbol table of the function because list/dict comprehensions have their own scope (see https://bugs.python.org/msg348274 for instance).
To get the output you expected, you just have to pass the local symbol table of the function to eval.
def bar():
one = '1'
two = '2'
three = '3'
func_locals = locals() # bind the locals() here
print({name: eval(name, globals(), func_locals) for name in ('one', 'two', 'three')})
bar()
Output
{'one': '1', 'two': '2', 'three': '3'}
I have some code and do not understand why the difference occurs:
np.std() which default ddof=0,when it's used alone.
but why when it's used as an argument in pivot_table(aggfunc=np.std),it changes into ddof=1 automatically.
import numpys as np
import pandas as pd
dft = pd.DataFrame({'A': ['one', 'one'],
'B': ['A', 'A'],
'C': ['bar', 'bar'],
'D': [-0.866740402,1.490732028]})
np.std(dft['D'])
#equivalent:np.std([-0.866740402,1.490732028]) (which:defaualt ddof=0)
#the result: 1.178736215
dft.pivot_table(index=['A', 'B'],columns='C',aggfunc=np.std)
#equivalent:np.std([-0.866740402,1.490732028],ddof=1)
#the result:1.666985
pivot uses DataFrame.groupby.agg and when you supply an aggregation function it's going to try to figure out exactly how to _aggregate.
arg=np.std will get handled here, the relevant code being
f = self._get_cython_func(arg)
if f and not args and not kwargs:
return getattr(self, f)(), None
Hidden in the DataFrame class is this table:
pd.DataFrame()._cython_table
#OrderedDict([(<function sum>, 'sum'),
# (<function max>, 'max'),
# ...
# (<function numpy.std>, 'std'),
# (<function numpy.nancumsum>, 'cumsum')])
pd.DataFrame()._cython_table.get(np.std)
#'std'
And so np.std is only used to select the attribute to call, the default ddof are completely ignored, and instead the pandas default of ddof=1 is used.
getattr(dft['D'], 'std')()
#1.6669847417133286
I'm not sure if the title of my question is the right description to the issue I'm facing.
I'm reading the following table of data from a spreadsheet and passing it as a dataframe:
Name Description Value
foo foobar 5
baz foobaz 4
bar foofoo 8
I need to transform this table of data to json following a specific schema.
I'm trying to get the following output:
{'global': {'Name': 'bar', 'Description': 'foofoo', 'spec': {'Value': '8'}}
So far I'm able to get the global and spec objects but I'm not sure how I should combine them to get the expected output above.
I wrote this:
for index, row in df.iterrows():
if row['Description'] == 'foofoo':
global = row.to_dict()
spec = row.to_dict()
del(global['Value'])
del(spec['Name'])
del(spec['Description'])
print("global:", global)
print("spec:", spec)
with the following output:
global: {'Name': 'bar', 'Description': 'foofoo'}
spec: {'Value': '8'}
How can I combine these two objects to get to the desired output?
This should give you that output:
global['spec'] = spec
combined = {'global': global}
Try this and see if it works faster: slow speed might be due to iterrows. I suggest you move the iteration to the dictionary after exporting from the dataframe.
Name Description Value
0 foo foobar 5
1 baz foobaz 4
2 bar foofoo 8
#Export dataframe to dictionar, using the 'index' option
M = df.to_dict('index')
r = {}
q = []
#iterating through the dictionary items(key,value pair)
for i,j in M.items():
#assign value to key 'global'
r['global'] = j
#popitem() works similarly to pop in list,
#take out the last item
#and remove it from parent dictionary
#this nests the spec key, inside the global key
r['global']['spec'] = dict([j.popitem()])
#this ensures the dictionaries already present are not overriden
#you could use copy or deep.copy to ensure same state
q.append(dict(r))
{'global': {'Name': 'foo', 'Description': 'foobar', 'spec': {'Value': 5}}}
{'global': {'Name': 'baz', 'Description': 'foobaz', 'spec': {'Value': 4}}}
{'global': {'Name': 'bar', 'Description': 'foofoo', 'spec': {'Value': 8}}}
dict popitem
I am trying to write some code that involves creating a default dictionary of dictionaries. However, I have no idea how to initialise/create such a thing. My current attempt looks something like this:
from collections import defaultdict
inner_dict = {}
dict_of_dicts = defaultdict(inner_dict(int))
The use of this default dict of dictionaries is to for each pair of words that I produce from a file I open (e.g. [['M UH M', 'm oo m']] ), to set each segment of the first word delimited by empty space as a key in the outer dictionary, and then for each segment in the second word delimited by empty space count the frequency of that segment.
For example
[['M UH M', 'm oo m']]
(<class 'dict'>, {'M': {'m': 2}, 'UH': {'oo': 1}})
Having just run this now it doesn't seem to have output any errors, however I was just wondering if something like this will actually produce a default dictionary of dictionaries.
Apologies if this is a duplicate, however previous answers to these questions have been confusing and in a different context.
To initialise a defaultdict that creates dictionaries as its default value you would use:
d = defaultdict(dict)
For this particular problem, a collections.Counter would be more suitable
>>> from collections import defaultdict, Counter
>>> d = defaultdict(Counter)
>>> for a, b in zip(*[x.split() for x in ['M UH M', 'm oo m']]):
... d[a][b] += 1
>>> print(d)
defaultdict(collections.Counter,
{'M': Counter({'m': 2}), 'UH': Counter({'oo': 1})})
Edit
You expressed interest in a comment about the equivalent without a Counter. Here is the equivalent using a plain dict
>>> from collections import defaultdict
>>> d = defaultdict(dict)
>>> for a, b in zip(*[x.split() for x in ['M UH M', 'm oo m']]):
... d[a][b] = d[a].get(b, 0) + 1
>>> print(d)
defaultdict(dict, {'M': {'m': 2}, 'UH': {'oo': 1}})
You also could a use a normal dictionary and its setdefault method.
my_dict.setdefault(key, default) will look up my_dict[key] and ...
... if the key already exists, return its current value without modifying it, or ...
... assign the default value (my_dict[key] = default) and then return that.
So you can call my_dict.setdefault(key, {}) always when you want to get a value from your outer dictionary instead of the normal my_dict[key] to retrieve either the real value assigned with this key if it#s present, or to get a new empty dictionary as default value which gets automatically stored into your outer dictionary as well.
Example:
outer_dict = {"M": {"m": 2}}
inner_dict = d.setdefault("UH", {})
# outer_dict = {"M": {"m": 2}, "UH": {}}
# inner_dict = {}
inner_dict["oo"] = 1
# outer_dict = {"M": {"m": 2}, "UH": {"oo": 1}}
# inner_dict = {"oo": 1}
inner_dict = d.setdefault("UH", {})
# outer_dict = {"M": {"m": 2}, "UH": {"oo": 1}}
# inner_dict = {"oo": 1}
inner_dict["xy"] = 3
# outer_dict = {"M": {"m": 2}, "UH": {"oo": 1, "xy": 3}}
# inner_dict = {"oo": 1, "xy": 3}
This way you always get a valid inner_dict, either an empty default one or the one that's already present for the given key. As dictionaries are mutable data types, modifying the returned inner_dict will also modify the dictionary inside outer_dict.
The other answers propose alternative solutions or show you can make a default dictionary of dictionaries using d = defaultdict(dict)
but the question asked how to make a default dictionary of default dictionaries, my navie first attempt was this:
from collections import defaultdict
my_dict = defaultdict(defaultdict(list))
however this throw an error: *** TypeError: first argument must be callable or None
so my second attempt which works is to make a callable using the lambda key word to make an anonymous function:
from collections import defaultdict
my_dict = defaultdict(lambda: defaultdict(list))
which is more concise than the alternative method using a regular function:
from collections import defaultdict
def default_dict_maker():
return defaultdict(list)
my_dict = defaultdict(default_dict_maker)
you can check it works by assigning:
my_dict[2][3] = 5
my_dict[2][3]
>>> 5
or by trying to return a value:
my_dict[0][0]
>>> []
my_dict[5]
>>> defaultdict(<class 'list'>, {})
tl;dr
this is your oneline answer my_dict = defaultdict(lambda: defaultdict(list))