Can anyone give me some examples of Python open-source projects using the function annotations introduced in Python 3?
I want to see some practical uses of this feature and see if I can use it my own project.
I have never seen this feature used in the wild. However, one potential use of function annotations that I explored in an article on Python 3 that I wrote for USENIX ;Login: was for enforcing contracts. For example, you could do this:
from functools import wraps
def positive(x):
'must be positive'
return x > 0
def negative(x):
'must be negative'
return x < 0
def ensure(func):
'Decorator that enforces contracts on function arguments (if present)'
return_check = func.__annotations__.get('return',None)
arg_checks = [(name,func.__annotations__.get(name))
for name in func.__code__.co_varnames]
#wraps(func)
def assert_call(*args):
for (name,check),value in zip(arg_checks,args):
if check:
assert check(value),"%s %s" % (name, check.__doc__)
result = func(*args)
if return_check:
assert return_check(result),"return %s" % (return_check.__doc__)
return result
return assert_call
# Example use
#ensure
def foo(a:positive, b:negative) -> positive:
return a-b
If you do this, you'll see behavior like this:
>>> foo(2,-3)
5
>>> foo(2,3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "ensure.py", line 22, in assert_call
assert check(value),"%s %s" % (name, check.__doc__)
AssertionError: b must be negative
I should note that the above example needs to be fleshed out more to work properly with default arguments, keyword arguments, and other details. It's only a sketch of an idea.
Now, whether or not this is a good idea or not, I just don't know. I'm inclined to agree with Brandon that the lack of composability is a problem--especially if annotations start to be used by different libraries for different purposes. I also wonder if something like this contract idea couldn't be accomplished through decorators instead. For example, making a decorator that was used like this (implementation left as an exercise):
#ensure(a=positive,b=negative)
def foo(a,b):
return a-b
A historial note, I've always kind of felt that function annotations were an outgrowth of discussions about "optional static typing" that the Python community had more than 10 years ago. Whether that was the original motivation or not, I just don't know.
I will play the curmudgeon, and recommend against using the feature. Hopefully it will someday be removed. Python has so far done a great job of deploying features that are attractive, orthogonal, and that can be stacked — function decorators are a great example: if I use three different libraries that all want me to decorate my function, the result looks rather clean:
#lru_cache(max_items=5)
#require_basic_auth
#view('index.html')
def index(…):
⋮
But this newfangled awkward “annotations” feature takes Python in the opposite direction: because you can only annotate a given function exactly once, it breaks completely the ability to compose solutions out of various pieces. If you had two libraries that each wanted you to annotate the same function on their behalf, then, from what I can see, you would be completely stymied.
Related
PyCharm Version: 2019.1.2
Python Version: 3.7
I am trying to use least code to reproduce the problem. And this is the snippet of my code:
def sql_reader():
def outer(func):
def wrapped_function(*args, **kwargs):
func(*args, **kwargs)
return [{"a": 1, "b": 2}]
return wrapped_function
return outer
#sql_reader()
def function_read():
return "1"
result = function_read()
for x in result:
print(x['a'])
print(result)
Basically, what I am doing is to "decorate" some function to output different types. For example, in this snippet, the function being decorated is returning 1 which is int. However, in decorator, I change the behavior and return list of dict.
Functionally speaking, it works fine. But it looks like my IDE always complains about it which is annoying as below:
Is there anyway I can get rid of this warning message?
With all due respect, you are using an over 3 year old version of PyCharm. I struggle to see a reason for this. The community edition is free and requires no root privileges to unpack and run on Linux systems. You should seriously consider upgrading to the latest version.
Same goes for Python by the way. You can install any version (including the latest) via Pyenv without root privileges. Although the Python version requirement may be subject to external restrictions for the code you are working on, so that is just a suggestion. But for the IDE I see no reason to use such an outdated version.
Since I am not using your PyCharm version, I can not reproduce your problem. Version 2022.2.3 has no such issues with your code. Be that as it may, there are a few things you can do to make life easier for static type checkers (and by extension for yourself).
The first thing I would always suggest is to use functools.wraps, when you are wrapping functions via a decorator. This preserves a lot of useful metadata about the wrapped function and even stores a reference to it in the wrapper's __wrapped__ attribute.
The second is proper type annotations. This should go for any code you write, unless it really is just a quick-and-dirty draft script that you will only use once and then throw away. The benefits of proper annotations especially in conjunction with modern IDEs are huge. There are many resources out there explaining them, so I won't go into details here.
In this concrete case, proper type hints will remove ambiguity about the return types of your functions and should work with any IDE (bugs non withstanding). In my version of PyCharm the return type of your wrapper function is inferred to be Any because no annotations are present, which prevents any warning like yours to pop up, but also doesn't allow any useful auto-suggestions to be provided.
Here is what I would do with your code: (should be compatible with Python 3.7)
from functools import wraps
from typing import Any, Callable, Dict, List
AnyFuncT = Callable[..., Any]
ResultsT = List[Dict[str, int]]
def sql_reader() -> Callable[[AnyFuncT], Callable[..., ResultsT]]:
def outer(func: AnyFuncT) -> Callable[..., ResultsT]:
#wraps(func)
def wrapper(*args: Any, **kwargs: Any) -> ResultsT:
func(*args, **kwargs)
return [{"a": 1, "b": 2}]
return wrapper
return outer
#sql_reader()
def function_read() -> str:
return "1"
Adding reveal_type(function_read()) underneath and calling mypy on this file results in the following:
note: Revealed type is "builtins.list[builtins.dict[builtins.str, builtins.int]]"
Success: no issues found in 1 source file
As you can see, at least mypy now correctly infers the type returned by the wrapper function we put around function_read. Your IDE should also correctly infer the types involved, but as I said I cannot verify this with my version.
Moreover, now PyCharm will give you auto-suggestions for methods available on the types involved:
results = function_read()
first = results[0]
value = first["a"]
If I now start typing results., PyCharm will suggest things like append, extend etc. because it recognizes result as a list. If I type first., it will suggest keys, values etc. (inferring it as a dictionary) and if I type value. it will give options like imag, real and to_bytes, which are available for integers.
More information: typing module docs
I came across an issue and I would love to get your help.
I am working with a certain codebase at work, and came across a function which returns 2 values, let's call this function my_func.
In some other function, my_func's output is being unpacked, for example:
a,b = my_func()
The thing is, I want to expand my_func to return 3 values instead of 2, however my_func is sort of an "interface function" which means that it is implemented by a lot of classes and I'm trying to avoid going to each of these classes and extend it's implementation directly.
I was wondering if there's something I can do to get the following results:
a,b,c = my_func()
without getting any unpacking exception. I would like if possible that c will get the value None from the "old implementation" classes.
Is this a good / bad practice? would love to get your opinion and help.
This question already has an answer here:
How do I detect and invoke a function when a python enum member is accessed
(1 answer)
Closed 2 years ago.
So, I've got a library which can read a certain file (file layout is out of my control). These files have versions, and with each version more features are added. I've got Enums that represent some features. Now with the latest update, new entries should be added to the Enum. But, I want to be able to give the user a warning (or exception) when they try to use newer features in older versions.
I figured decorators might be the way to go, marking certain values & functions like: #from_version('1.40') or something like it. I've never created my own decorators so I'm not too familiar with them.
One final problem with this, the version of the file isn't globally available. (I can't add something static-ish because multiple files should be able to be opened at once). So I'm not even sure if decorators are the way to go.
Now I tried adding a decorator to an Enum entry, but that didn't work:
import enum
>>> def hello_world(f):
... def decorated(*args, **kwargs):
... print('Hello World!')
... return decorated
...
>>> class A(enum.Enum):
... #hello_world() # Tried with and without ()
... ABC = 1
File "<input>", line 3
ABC = 1
^
SyntaxError: invalid syntax
So I'm wondering how I can add this to an enum, I also tried adding a property in an enum so I can add the decorator to that, but even adding the property didn't work...
TL;DR
How do I add my own decorator to a specific enum entry? I expected it to look like this:
class EnumClass(Enum):
#from_version('1.40')
ENTRY = 0
Thanks in advance for any help!
What you want is in a separate answer, but to explain the problem you are having -- decorators are only usable with functions and classes (def and class) and cannot be used with attributes (things like ABC = 1.
I have a python module which has several methods:
module.py
def a():
return "This is a method"
I want to add some additional functionality to method a by calling it from script to avoid modification in module itself.
How can I add additional operations or arguments to that module calling it from script?
For example I imported module.py in my script and in that script I add two arguments to method "a" and addition of those arguments in addition to:
return "This is a method"
Well, you cannot really change any imported methods as far as I am aware. That does not mean it is completely impossible, I just wouldn't know how.
Question is though if that is actually what you want (as I believe it is a very uncommon way of handling code and I am guessing not an advisable one)?
Could it be that what you actually want to achieve is something more like this:
module_script.py
def a(variable):
return_value = "This is a " + str(variable) + " method!"
return return_value
second_script.py
import module_script
print(module_script.a("nice"))
The output of the second script would be:
"This is a nice method!"
I have a subparser named app01. It has two arguments --delete-user, and --delete-hash. I've set the default function to call the function check_app
the check_app function is something like this:
def check_app(args):
a = App()
if args.user:
# run function01 from App() class to delete user account
elif args.hash:
# run function02 from App() class delete user hash
else:
# run both functions to delete user and hash
How do i get that to work without having to add both function01 and function02 into the else block. Or is that the only way?
I think what code you have is decent.
If you want what you want this for purely aesthetic reasons, let me warn you that the code maintainability will drop substantially. You can see the answer given by Ronan B already seems pretty confusing and if you're the person trying to understand it, it feels like too much work to decipher.
But if you still want me to titillate your senses, I may as well suggest a simpler version:
def check_app(args):
a = App()
if not args.user:
# run #02 < note #2
if not args.hash:
# run #01 < note #1
This is of course assuming that if both functions #1 and #2 need to run, no arguments are given, instead of both arguments being given, as is typically intuitive.