I'm looking for a tool that adds the type annotation that are already added in a function to the docstring generated by PyCharm.
I have been following this issue on JetBrains for a long time and no progress seem to be made on this ticket yet.
I saw this article as well but it seems too specific and only works with google docstrings.
I have checked out this package on PyPI but this does the opposite of what I need. What I need is to add the type hints provided in the argument to the docstring and not the opposite.
This might be related to How to make PyCharm get type hints from function definition and populate type values in docstrings?, but I just need a general purpose solution, or at least one that works with Numpy docstring or reStructuredText and maybe play nicely with PyCharm.
I just see myself coming back to researching this thing in the hopes of getting something but couldn't find what I needed. What tools would work for this?
When I was searching few months ago, I couldn't find any usable solution for this. Right now, I am using VS Code with this extension which gives nice Docstring:
def foo(a: int, b: str) -> bool:
"""[summary]
Args:
a (int): [description]
b (str): [description]
Returns:
bool: [description]
"""
Related
PyCharm Version: 2019.1.2
Python Version: 3.7
I am trying to use least code to reproduce the problem. And this is the snippet of my code:
def sql_reader():
def outer(func):
def wrapped_function(*args, **kwargs):
func(*args, **kwargs)
return [{"a": 1, "b": 2}]
return wrapped_function
return outer
#sql_reader()
def function_read():
return "1"
result = function_read()
for x in result:
print(x['a'])
print(result)
Basically, what I am doing is to "decorate" some function to output different types. For example, in this snippet, the function being decorated is returning 1 which is int. However, in decorator, I change the behavior and return list of dict.
Functionally speaking, it works fine. But it looks like my IDE always complains about it which is annoying as below:
Is there anyway I can get rid of this warning message?
With all due respect, you are using an over 3 year old version of PyCharm. I struggle to see a reason for this. The community edition is free and requires no root privileges to unpack and run on Linux systems. You should seriously consider upgrading to the latest version.
Same goes for Python by the way. You can install any version (including the latest) via Pyenv without root privileges. Although the Python version requirement may be subject to external restrictions for the code you are working on, so that is just a suggestion. But for the IDE I see no reason to use such an outdated version.
Since I am not using your PyCharm version, I can not reproduce your problem. Version 2022.2.3 has no such issues with your code. Be that as it may, there are a few things you can do to make life easier for static type checkers (and by extension for yourself).
The first thing I would always suggest is to use functools.wraps, when you are wrapping functions via a decorator. This preserves a lot of useful metadata about the wrapped function and even stores a reference to it in the wrapper's __wrapped__ attribute.
The second is proper type annotations. This should go for any code you write, unless it really is just a quick-and-dirty draft script that you will only use once and then throw away. The benefits of proper annotations especially in conjunction with modern IDEs are huge. There are many resources out there explaining them, so I won't go into details here.
In this concrete case, proper type hints will remove ambiguity about the return types of your functions and should work with any IDE (bugs non withstanding). In my version of PyCharm the return type of your wrapper function is inferred to be Any because no annotations are present, which prevents any warning like yours to pop up, but also doesn't allow any useful auto-suggestions to be provided.
Here is what I would do with your code: (should be compatible with Python 3.7)
from functools import wraps
from typing import Any, Callable, Dict, List
AnyFuncT = Callable[..., Any]
ResultsT = List[Dict[str, int]]
def sql_reader() -> Callable[[AnyFuncT], Callable[..., ResultsT]]:
def outer(func: AnyFuncT) -> Callable[..., ResultsT]:
#wraps(func)
def wrapper(*args: Any, **kwargs: Any) -> ResultsT:
func(*args, **kwargs)
return [{"a": 1, "b": 2}]
return wrapper
return outer
#sql_reader()
def function_read() -> str:
return "1"
Adding reveal_type(function_read()) underneath and calling mypy on this file results in the following:
note: Revealed type is "builtins.list[builtins.dict[builtins.str, builtins.int]]"
Success: no issues found in 1 source file
As you can see, at least mypy now correctly infers the type returned by the wrapper function we put around function_read. Your IDE should also correctly infer the types involved, but as I said I cannot verify this with my version.
Moreover, now PyCharm will give you auto-suggestions for methods available on the types involved:
results = function_read()
first = results[0]
value = first["a"]
If I now start typing results., PyCharm will suggest things like append, extend etc. because it recognizes result as a list. If I type first., it will suggest keys, values etc. (inferring it as a dictionary) and if I type value. it will give options like imag, real and to_bytes, which are available for integers.
More information: typing module docs
I'm really new to coding and was stuck at one of the Leetcode problems on Binary Tree Transversing.
Can anyone please explain what the following line of code means:
def preorderTraversal(self, root: Optional[TreeNode]) -> List[int]:
From what I understood, the core was trying to define a function pre-order traversal and a class TreeNode was defined earlier. However I was not able to get what root: Optional[TreeNode] and ->List[int] meant.
Thanks in advance
If it helps, you can ignore the type hints in that function declaration. If you change it to this, it will still work:
def preorderTraversal(self, root):
The type hints are helping editors (and you) to know the expected type of parameters and of the function's return value:
Optional: this means the value could be None
TreeNode: this means the value is an instance of the class TreeNode. LeetCode provides its definition in a comment block in the original code template you get to work with. It is indeed predefined.
List[int]: this means the returned value should be a list, and the list elements should be integers.
I am looking at Kedro Library as my team are looking into using it for our data pipeline.
While going to the offical tutorial - Spaceflight.
I came across this function:
def preprocess_companies(companies: pd.DataFrame) -> pd.DataFrame:
"""Preprocess the data for companies.
Args:
companies: Source data.
Returns:
Preprocessed data.
"""
companies["iata_approved"] = companies["iata_approved"].apply(_is_true)
companies["company_rating"] = companies["company_rating"].apply(_parse_percentage)
return companies
companies is the name of the csv file containing the data
Looking at the function, my assumption is that (companies: pd.Dafarame) is the shorthand to read the "companies" dataset as a dataframe. If so, I do not understand what does -> pd.Dataframe at the end means
I tried looking at python documentation regarding such style of code but I did not managed to find any
Much help is appreciated to assist me in understanding this.
Thank you
This is tht way of declaring type of your inputs(companies: pd.DataFrame) . Here comapnies is argument and pd.DataFrame is its type . in same way -> pd.DataFrame this is the type of output
Overall they are saying that comapnies of type pd.DataFrame will return pd.DataFrametype variable .
I hope you got it
The -> notation is type hinting, as is the : part in the companies: pd.DataFrame function definition. This is not essential to do in Python but many people like to include it. The function definition would work exactly the same if it didn't contain this but instead read:
def preprocess_companies(companies):
This is a general Python thing rather than anything kedro-specific.
The way that kedro registers companies as a kedro dataset is completely separate from this function definition and is done through the catalog.yml file:
companies:
type: pandas.CSVDataSet
filepath: data/01_raw/companies.csv
There will then a node defined (in pipeline.py) to specify that the preprocess_companies function should take as input the kedro dataset companies:
node(
func=preprocess_companies,
inputs="companies", # THIS LINE REFERS TO THE DATASET NAME
outputs="preprocessed_companies",
name="preprocessing_companies",
),
In theory the name of the parameter in the function itself could be completely different, e.g.
def preprocess_companies(anything_you_want):
... although it is very common to give it the same name as the dataset.
In this situation companies is technically any DataFrame. However, when wrapped in a Kedro Node object the correct dataset will be passed in:
Node(
func=preprocess_companies, # The function posted above
inputs='raw_companies', # Kedro will read from a catalog entry called 'raw companies'
outputs='processed_companies', # Kedro will write to a catalog entry called 'processed_companies'
)
In essence the parameter name isn't really important here, it has been named this way so that the person reading the code knows that it is semantically about companies, but the function name does that too.
The above is technically a simplification since I'm not getting into MemoryDataSets but hopefully it covers the main points.
def resize_img( size: Tuple[int, int] = (299, 299)):
pass
What does the parameter mean？I did not find the docs.
In case you are asking about Tuple[int, int]: this syntax is provided by typing module. It helps you and other people reading your code to understand which type should parameters have when passed into function. In your example - if you try to pass something different than tuple of two ints (i.e. resize_img(5)) IDE will mark it as Expected type 'Tuple[int]', got 'int' instead. This does not break code execution, but shows developer that probably he/she uses this function with wrong type of parameter passed in.
I've just started using PyCharm and it is teaching me quite a bit about the correct ways to document things. That documentation then makes the IDE more productive. But, it's raised a question for me.
I see that there are a couple of ways to document the data type of the value returned by a function.
In the function definition use -> type
In the function comments use :rtype: type
But, how does this work for functions that return multiple values? For example, if a function returns both a boolean AND a float, how do you document this?
Thanks!
It is really simple using type annotation:
def func(i: int) -> (bool, float):
You would do something similar in the docstring.