In brief, I have a DataFormatter class that has two possible states: train or infer, that should act similarly to many of the sklearn libraries that have fit and transform functions: if mode is train I want to store in self.metadata a list of the function calls and args that were made, so that they can simply be reapplied verbatim and in order at infer time.
So minimally, I have:
import inspect
class DataFormatter:
def __init__(self, mode, data=None):
self.data = data
self.metadata = []
# The decorator function: broken, but something like--
def meta(self, f, *args):
def wrapper(*args):
return f(*args)
if self.mode == 'train':
print('caching metadata')
meta = {
f.__name__: {
param: arg for param, arg in zip(
inspect.getfillargspec(f).args, args)}}
self.metadata.append(meta)
return wrapper
#meta
def drop(self, cols):
self.data = self.data.drop(cols)
Then if I use:
formatter = DataFormatter('train', my_data)
formatter.drop(['col1', 'col5'])
print(formatter.metadata)
...I would like to get:
[{'drop': {'cols': ['col1', 'col5']}}]
I have tried various permutations and placements of self, and pulling the decorator func outside the class altogether, but no luck so far.
#kindall says "You can't get self at decoration time because the decorator is applied at function definition time. No self exists yet; in fact, the class doesn't exist yet." (Possible to create a #synchronized decorator that's aware of a method's object?), so not even sure if this is possible...
What will "see" a self is the decorated function - and it is represented by the function you call "wrapper":
import inspect
# The decorator function: should be out of the class body
def meta( f): ,
def wrapper(self, *args):
# call the decorated function:
# (but you could run other code, including inspecting
# and modifying arguments, here as well)
result = f(self, *args)
# now, your original method had run, and you have
# access to self:
if self.mode == 'train':
print('caching metadata')
meta = {
f.__name__: {
param: arg for param, arg in zip(
inspect.getfillargspec(f).args, args)}}
self.metadata.append(meta)
return result
return wrapper
class DataFormatter:
def __init__(self, mode, data=None):
self.data = data
self.metadata = []
#meta
def drop(self, cols):
self.data = self.data.drop(cols)
That will work.
How do I call a function, using a string with the function's name? For example:
import foo
func_name = "bar"
call(foo, func_name) # calls foo.bar()
Given a module foo with method bar:
import foo
bar = getattr(foo, 'bar')
result = bar()
getattr can similarly be used on class instance bound methods, module-level methods, class methods... the list goes on.
Using locals(), which returns a dictionary with the current local symbol table:
locals()["myfunction"]()
Using globals(), which returns a dictionary with the global symbol table:
globals()["myfunction"]()
Based on Patrick's solution, to get the module dynamically as well, import it using:
module = __import__('foo')
func = getattr(module, 'bar')
func()
Just a simple contribution. If the class that we need to instance is in the same file, we can use something like this:
# Get class from globals and create an instance
m = globals()['our_class']()
# Get the function (from the instance) that we need to call
func = getattr(m, 'function_name')
# Call it
func()
For example:
class A:
def __init__(self):
pass
def sampleFunc(self, arg):
print('you called sampleFunc({})'.format(arg))
m = globals()['A']()
func = getattr(m, 'sampleFunc')
func('sample arg')
# Sample, all on one line
getattr(globals()['A'](), 'sampleFunc')('sample arg')
And, if not a class:
def sampleFunc(arg):
print('you called sampleFunc({})'.format(arg))
globals()['sampleFunc']('sample arg')
Given a string, with a complete python path to a function, this is how I went about getting the result of said function:
import importlib
function_string = 'mypackage.mymodule.myfunc'
mod_name, func_name = function_string.rsplit('.',1)
mod = importlib.import_module(mod_name)
func = getattr(mod, func_name)
result = func()
The best answer according to the Python programming FAQ would be:
functions = {'myfoo': foo.bar}
mystring = 'myfoo'
if mystring in functions:
functions[mystring]()
The primary advantage of this technique is that the strings do not need to match the names of the functions. This is also the primary technique used to emulate a case construct
The answer (I hope) no one ever wanted
Eval like behavior
getattr(locals().get("foo") or globals().get("foo"), "bar")()
Why not add auto-importing
getattr(
locals().get("foo") or
globals().get("foo") or
__import__("foo"),
"bar")()
In case we have extra dictionaries we want to check
getattr(next((x for x in (f("foo") for f in
[locals().get, globals().get,
self.__dict__.get, __import__])
if x)),
"bar")()
We need to go deeper
getattr(next((x for x in (f("foo") for f in
([locals().get, globals().get, self.__dict__.get] +
[d.get for d in (list(dd.values()) for dd in
[locals(),globals(),self.__dict__]
if isinstance(dd,dict))
if isinstance(d,dict)] +
[__import__]))
if x)),
"bar")()
For what it's worth, if you needed to pass the function (or class) name and app name as a string, then you could do this:
myFnName = "MyFn"
myAppName = "MyApp"
app = sys.modules[myAppName]
fn = getattr(app,myFnName)
Try this. While this still uses eval, it only uses it to summon the function from the current context. Then, you have the real function to use as you wish.
The main benefit for me from this is that you will get any eval-related errors at the point of summoning the function. Then you will get only the function-related errors when you call.
def say_hello(name):
print 'Hello {}!'.format(name)
# get the function by name
method_name = 'say_hello'
method = eval(method_name)
# call it like a regular function later
args = ['friend']
kwargs = {}
method(*args, **kwargs)
As this question How to dynamically call methods within a class using method-name assignment to a variable [duplicate] marked as a duplicate as this one, I am posting a related answer here:
The scenario is, a method in a class want to call another method on the same class dynamically, I have added some details to original example which offers some wider scenario and clarity:
class MyClass:
def __init__(self, i):
self.i = i
def get(self):
func = getattr(MyClass, 'function{}'.format(self.i))
func(self, 12) # This one will work
# self.func(12) # But this does NOT work.
def function1(self, p1):
print('function1: {}'.format(p1))
# do other stuff
def function2(self, p1):
print('function2: {}'.format(p1))
# do other stuff
if __name__ == "__main__":
class1 = MyClass(1)
class1.get()
class2 = MyClass(2)
class2.get()
Output (Python 3.7.x)
function1: 12
function2: 12
none of what was suggested helped me. I did discover this though.
<object>.__getattribute__(<string name>)(<params>)
I am using python 2.66
Hope this helps
Although getattr() is elegant (and about 7x faster) method, you can get return value from the function (local, class method, module) with eval as elegant as x = eval('foo.bar')(). And when you implement some error handling then quite securely (the same principle can be used for getattr). Example with module import and class:
# import module, call module function, pass parameters and print retured value with eval():
import random
bar = 'random.randint'
randint = eval(bar)(0,100)
print(randint) # will print random int from <0;100)
# also class method returning (or not) value(s) can be used with eval:
class Say:
def say(something='nothing'):
return something
bar = 'Say.say'
print(eval(bar)('nice to meet you too')) # will print 'nice to meet you'
When module or class does not exist (typo or anything better) then NameError is raised. When function does not exist, then AttributeError is raised. This can be used to handle errors:
# try/except block can be used to catch both errors
try:
eval('Say.talk')() # raises AttributeError because function does not exist
eval('Says.say')() # raises NameError because the class does not exist
# or the same with getattr:
getattr(Say, 'talk')() # raises AttributeError
getattr(Says, 'say')() # raises NameError
except AttributeError:
# do domething or just...
print('Function does not exist')
except NameError:
# do domething or just...
print('Module does not exist')
In python3, you can use the __getattribute__ method. See following example with a list method name string:
func_name = 'reverse'
l = [1, 2, 3, 4]
print(l)
>> [1, 2, 3, 4]
l.__getattribute__(func_name)()
print(l)
>> [4, 3, 2, 1]
Nobody mentioned operator.attrgetter yet:
>>> from operator import attrgetter
>>> l = [1, 2, 3]
>>> attrgetter('reverse')(l)()
>>> l
[3, 2, 1]
>>>
getattr calls method by name from an object.
But this object should be parent of calling class.
The parent class can be got by super(self.__class__, self)
class Base:
def call_base(func):
"""This does not work"""
def new_func(self, *args, **kwargs):
name = func.__name__
getattr(super(self.__class__, self), name)(*args, **kwargs)
return new_func
def f(self, *args):
print(f"BASE method invoked.")
def g(self, *args):
print(f"BASE method invoked.")
class Inherit(Base):
#Base.call_base
def f(self, *args):
"""function body will be ignored by the decorator."""
pass
#Base.call_base
def g(self, *args):
"""function body will be ignored by the decorator."""
pass
Inherit().f() # The goal is to print "BASE method invoked."
i'm facing the similar problem before, which is to convert a string to a function. but i can't use eval() or ast.literal_eval(), because i don't want to execute this code immediately.
e.g. i have a string "foo.bar", and i want to assign it to x as a function name instead of a string, which means i can call the function by x() ON DEMAND.
here's my code:
str_to_convert = "foo.bar"
exec(f"x = {str_to_convert}")
x()
as for your question, you only need to add your module name foo and . before {} as follows:
str_to_convert = "bar"
exec(f"x = foo.{str_to_convert}")
x()
WARNING!!! either eval() or exec() is a dangerous method, you should confirm the safety.
WARNING!!! either eval() or exec() is a dangerous method, you should confirm the safety.
WARNING!!! either eval() or exec() is a dangerous method, you should confirm the safety.
You means get the pointer to an inner function from a module
import foo
method = foo.bar
executed = method(parameter)
This is not a better pythonic way indeed is possible for punctual cases
This is a simple answer, this will allow you to clear the screen for example. There are two examples below, with eval and exec, that will print 0 at the top after cleaning (if you're using Windows, change clear to cls, Linux and Mac users leave as is for example) or just execute it, respectively.
eval("os.system(\"clear\")")
exec("os.system(\"clear\")")
Please help me solivng problems with output sentence contains apostrophe ‘ symbol wit standard format() function. I take example from chapter 31 «Learning Python, Fifth Edition» by Mark Lutz 2013 with function, instance, method and instance with print overloading — see code and theirs outout below . Interpreter information - Python 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0] on linux .
Example code and output with format() and print() functions:
def square(arg): # Simple functions def or lambda
return arg ** 2
class Sum: # Calliable instance
def __init__(self, val):
self.val = val
def __call__(self, arg):
return self.val + arg
class Product: # Bound method
def __init__(self, val):
self.val = val
def method(self, arg):
return self.val * arg
class Negate:
def __init__(self, val): # Operator print overloading
self.val = -val
def __repr__(self):
return str(self.val)
if __name__ == '__main__':
Sobject = Sum(2)
pobject = Product(3)
actions = [square, Sobject, pobject.method, Negate] # function, instance, method, instance with print() overloading
table = {act(5): act for act in actions} # Dictionary comprhension
for (key, value) in table.items(): # Printing with format
print('{0:2} => {1}'.format(key, value))
output print correct first three rows and raise exceptions when trying print fourth row which contains mean (-5, <class '__main__.Negate'>) with apostrophe ' symbol
25 => <function square at 0x7fdba0d64840>
7 => <__main__.Sum object at 0x7fdba00a8d30>
15 => <bound method Product.method of <__main__.Product object at 0x7fdba00a8d68>>
Traceback (most recent call last): TypeError: unsupported format string passed to Negate.__format__
print(table) # using standard print() function
output is correct without any errors, see bellow:
dict_items(
[(25, <function square at 0x7fdba0088d90>),
(7, <__main__.Sum object at 0x7fdba00215c0>),
(15, <bound method Product.method of <__main__.Product object at 0x7fdba0021390>>),
(-5, <class '__main__.Negate'>)])
I know how print apostrophe ‘ inside standard print() function - add the escape character ' to add an apostrophe in a string that is enclosed in single quotes or replace single quotes to double quotes:
print('I\'m studying Python') → I'm studing Python
or
print("I'm studying Python") -> I'm studying Python
How fix this problem with apostrophe ' symbol inside format() function?
to Pranav Hosangadi
Example takes from Chapter 31: "Designing with Classes" pages 952 - 953 - class Negate has only constructor with def init(self, val) and overloading print() with def repr(self): and hasn't any additional overloading for format and other embedded method for objects described in this example. This example in the book has correct output -5 => <class 'main.Negate'> on the page 953
I get the following error from my code: TypeError: get_manifest() takes 0 positional arguments but 1 was given
I have a class scan, and get_manifest is a function in scan:
import amfy
import requests
import json
from datetime import timedelta, datetime
class Scan:
def __init__(self, realm = 416, cluster = 9):
self.events = list()
self.url = "http://realm{}.c{}0.castle.rykaiju.com/api/locales/en/get_serialized_new".format(realm, cluster)
def get_manifest():
self.request = requests.get(self.url)
if self.request.status_code == 200:
self.manifest = self.request.json() if "json" in self.request.headers["content-type"] else amfy.loads(self.request.content)
else:
self.manifest = {"failed": req.reason}
def get_time(day):
self.today = datetime.now()
self.plusdays = timedelta(days = day)
self.new_date = sel.ftoday + self.plusdays
self.year, self.month, self.day = self.new_date.year, self.new_date.month, self.new_date.day
if len(str(self.day)) == 1:
day = "0{}".format(self.day)
if len(str(self.month)) == 1:
month = "0{}".format(self.month)
self.date = str(self.year) + str(self.month) + str(self.day)
return self.date
def search_events():
for day in range(0, 11):
date = self.get_time(day)
for section in doaManifest:
for key, value in doaManifest[section].items():
if date in key:
event_title = key
try:
event_desc = value['text']
except KeyError:
event_desc = """" /!\ No Event Description /!\""""
my_dict = {'title' : event_title, 'contents' : event_desc}
self.events.append(my_dict)
Then, in another class, which is my app GUI (written with tkinter), I have a button that calls on this class. The button's command is this:
def scan(self):
if self.scan['text'] == 'Scan':
self.scan['text'] = 'Re-Scan'
self.scanned = True
self.scan = Scan()
self.scan.get_manifest()
self.scan.search_events()
I don't feed get_manifest any arguments, so why is it saying I am?
Seems like you forgot adding self to get_manifest(). Redefine get_manifest() like so:
def get_manifest(self):
code...
Instance methods should always accept self as the first argument as it's automatically inserted. You may use #staticmethod if you don't wish to receive it but as you're using self I guess that it's not something you want.
First you need to add the self parameter to all class methods unless you use #staticmethod. So your class should look like this:
class Scan:
def __init__(self, realm = 416, cluster = 9):
# code...
def get_manifest(self):
# code...
def get_time(self, day):
# code...
def search_events(self):
# code...
Furthermore, to use this class within your scan() method you need to initialize and save the class instance to a variable, then call on the get_manifest() method from the saved instance variable.
def scan(self):
# I assume this scan() method is actually a part of a class since 'self' is used
if self.scan['text'] == 'Scan':
self.scan['text'] = 'Re-Scan'
self.scanned = True
#self.scan = Scan() # You are overwriting the definition of 'scan' here...
# So instead do the below
# Initialize and save your Scan() instance to a variable
scan_instance = Scan()
# Now you may use the object's methods
scan_instance.get_manifest()
scan_instance.search_events()
# Here you can now access the object's attributes that were created in get_manifest()
print(scan_instance.manifest)
I've tried to write a decorator function which wraps an asyncio.coroutine and returns the time it took to get done. The recipe below contains the code which is working as I expected. My only problem with it that somehow I loose the name of the decorated function despite the use of #functools.wraps. How to retain the name of the original coroutine? I checked the source of asyncio.
import asyncio
import functools
import random
import time
MULTIPLIER = 5
def time_resulted(coro):
#functools.wraps(coro)
#asyncio.coroutine
def wrapper(*args, **kargs):
time_before = time.time()
result = yield from coro(*args, **kargs)
if result is not None:
raise TypeError('time resulted coroutine can '
'only return None')
return time_before, time.time()
print('= wrapper.__name__: {!r} ='.format(wrapper.__name__))
return wrapper
#time_resulted
#asyncio.coroutine
def random_sleep():
sleep_time = random.random() * MULTIPLIER
print('{} -> {}'.format(time.time(), sleep_time))
yield from asyncio.sleep(sleep_time)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
tasks = [asyncio.Task(random_sleep()) for i in range(5)]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()
for task in tasks:
print(task, task.result()[1] - task.result()[0])
print('= random_sleep.__name__: {!r} ='.format(
random_sleep.__name__))
print('= random_sleep().__name__: {!r} ='.format(
random_sleep().__name__))
The result:
= wrapper.__name__: 'random_sleep' =
1397226479.00875 -> 4.261069174838891
1397226479.00875 -> 0.6596335046471768
1397226479.00875 -> 3.83421163259601
1397226479.00875 -> 2.5514027672929713
1397226479.00875 -> 4.497471439365472
Task(<wrapper>)<result=(1397226479.00875, 1397226483.274884)> 4.266134023666382
Task(<wrapper>)<result=(1397226479.00875, 1397226479.6697)> 0.6609499454498291
Task(<wrapper>)<result=(1397226479.00875, 1397226482.844265)> 3.835515022277832
Task(<wrapper>)<result=(1397226479.00875, 1397226481.562422)> 2.5536720752716064
Task(<wrapper>)<result=(1397226479.00875, 1397226483.51523)> 4.506479978561401
= random_sleep.__name__: 'random_sleep' =
= random_sleep().__name__: 'wrapper' =
As you can see random_sleep() returns a generator object with different name. I would like to retain the name of the decorated coroutine. I am not aware if this is problem is specific to asyncio.coroutines or not. I also tried the code with different decorator orders, but all has the same result. If I comment #functools.wraps(coro) then even random_sleep.__name__ becomes wrapper as I expected.
EDIT: I've posted this issue to Python Issue Tracker and received the following answer by R. David Murray: "I think this is a specific case of a more general need to improve 'wraps' that was discussed on python-dev not too long ago."
The issue is that functools.wraps changes only wrapper.__name__ and wrapper().__name__ stays wrapper. __name__ is a readonly generator attribute. You could use exec to set appropriate name:
import asyncio
import functools
import uuid
from textwrap import dedent
def wrap_coroutine(coro, name_prefix='__' + uuid.uuid4().hex):
"""Like functools.wraps but preserves coroutine names."""
# attribute __name__ is not writable for a generator, set it dynamically
namespace = {
# use name_prefix to avoid an accidental name conflict
name_prefix + 'coro': coro,
name_prefix + 'functools': functools,
name_prefix + 'asyncio': asyncio,
}
exec(dedent('''
def {0}decorator({0}wrapper_coro):
#{0}functools.wraps({0}coro)
#{0}asyncio.coroutine
def {wrapper_name}(*{0}args, **{0}kwargs):
{0}result = yield from {0}wrapper_coro(*{0}args, **{0}kwargs)
return {0}result
return {wrapper_name}
''').format(name_prefix, wrapper_name=coro.__name__), namespace)
return namespace[name_prefix + 'decorator']
Usage:
def time_resulted(coro):
#wrap_coroutine(coro)
def wrapper(*args, **kargs):
# ...
return wrapper
It works but there is probably a better way than using exec().
In the time since this question was asked, it became possible to change the name of a coroutine. It is done by setting __qualname__ (not __name__):
async def my_coro(): pass
c = my_coro()
print(repr(c))
# <coroutine object my_coro at 0x7ff8a7d52bc0>
c.__qualname__ = 'flimflam'
print(repr(c))
# <coroutine object flimflam at 0x7ff8a7d52bc0>
import asyncio
print(repr(asyncio.ensure_future(c)))
# <Task pending name='Task-737' coro=<flimflam() running at <ipython-input>:1>>
The usage of __qualname__ in a coroutine object's __repr__ is defined in the CPython source