The following code works as expected, but the os.path.join produces a type error using pyright in VSCode, where shown.
# python 3.6.9
# pyright 1.1.25
# windows 10
# vscode 1.42.1
import os
import tempfile
with tempfile.TemporaryDirectory() as tmpfolder:
name = "hello.txt"
path = os.path.join(tmpfolder, name)
# No overloads for 'os.path.join(tmpfolder, name)' match parameters
# Argument types: (TypeVar['AnyStr', str, bytes], Literal['hello.txt'])
print(path)
I think I understand the immediate cause of the problem, but contend it should not be happening. Given that, I have some questions:
Is this the idiomatic way to write this code?
Is the problem in tempfile, os, pyright, or me?
If I cannot upgrade Python, what is the best (i.e. least clunky) way to suppress the error?
This seems like a limitation of pyright.
In short, the tempfile.TemporaryDirectory class is typed to be generic with respect to AnyStr. However, your example code omits specifying the generic type, leaving it up to the type checker to infer something appropriate.
In this case, I think there are several reasonable things for a type checker to do:
Pick some default generic type based on the typevar, such as 'str' or 'Union[str, bytes]'. For example, mypy ends up picking 'str' by default, giving 'tmpfolder' a type of 'str'.
Pick some placeholder type like either 'Any', the dynamic type, or NoReturn (aka 'bottom' aka 'nothing'). Both types are a valid subtype of every type, so are guaranteed to be valid placeholders and not cause downstream errors. This is what pyre and pytype does -- they deduce 'tmpfolder' has type 'Any' and 'nothing' respectively.
Attempt to infer the correct type based on context. Some type checkers may attempt to do this, but I don't know of any that handles this particular case perfectly.
Report an error and ask the user to specify the desired generic type.
What pyright seems to do instead is to just "leak" the generic variable. There is perhaps a principled reason why pyright is deciding to do this that I'm overlooking, but IMO this seems like a bug.
To answer your other questions, your example program is idiomatic Python, and type checkers should ideally support it without modification.
Adding a # type: ignore comment to the line with the error is the PEP 484 sanctioned way of suppressing error messages. I'm not familiar enough with pyright to know if it has a different preferred way of suppressing errors.
Related
Is there a neat solution to raise an error if a value is passed to the NamedTuple field that does not match the declared type?
In this example, I intentionally passed page_count str instead of int. And the script will work on passing the erroneous value forward.
(I understand that linter will draw your attention to the error, but I encountered this in a case where NamedTuple fields were filled in by a function getting values from config file).
I could check the type of each value with a condition, but it doesn't look really clean. Any ideas? Thanks.
from typing import NamedTuple
class ParserParams(NamedTuple):
api_url: str
page_count: int
timeout: float
parser_params = ParserParams(
api_url='some_url',
page_count='3',
timeout=10.0,
)
By design, Python is a dynamically typed language which means any value can be assigned to any variable. Typing is only supported as hints - the errors might be highlighted in your IDE, but they do not enforce anything.
This means that if you need type checking you have to implement it yourself. On the upside, this can probably be automated, i.e. implemented only once instead of separately for every field. However, NamedTuple does not provide such checking out of the box.
I've been using Python's type annotations in an unusual way: I have some code that inspects the annotations of a function's arguments at run-time, searches for values that match the types of the arguments, and calls the function with values of appropriate types (if found).
This has been working wonderfully, and was remarkably easy to implement, but the source file has grown fairly large, so today I tried breaking it into multiple files. I found that get_type_hints() can't evaluate a type annotation if it's from a module other than the current one, at least not without being given a global namespace where all the needed types are defined.
I'm using from __future__ import annotations everywhere, so the type annotations in each function's .__annotations__ attribute are stored as strings in need of evaluation. To evaluate them, I need the globals from the module where the function was defined. How can I get that? Or will that even work? I'm using if TYPE_CHECKING: to avoid circular imports; consequently some annotations won't be available at run-time in each module where they're applied to a function.
Here's the code that extracts the types of the function arguments, if that helps:
def params_of(func: Callable) -> Iterable[Tuple[str, TypeAnnotation]]:
type_hints = get_type_hints(func)
for param_name in inspect.signature(func).parameters:
if param_name == 'return':
continue # disregard return type
yield (param_name, type_hints.get(param_name, Any))
(TypeAnnotation is only for readability; it's defined to Any.)
Is it possible to instruct GHC compiler to require that a specific value in code has invalid type, without ever using this value?
A contrived example is:
data Box a = Num a => Box a
goodBoxSample :: Box Int
goodBoxSample = Box 1
-- below definition and binding are expected to fail compilation
badBoxSample :: Box String
badBoxSample = Box "foo"
Is there a way to inform the compiler that badBoxSample is expected to fail (e.g. with some pragma, rather than commenting it out as a known bad sample), so that the code compiles only if badBoxSample fails to type-check?
The motivation here is the same as for writing a test (in some other language) with the code that is required to throw exception for the test case to pass.
Not possible. You're basically asking for a way to prove that there's no instance Num String, but Haskell operates under the open-world assumption, which means that someone could always declare such an instance.
Somebody posted a very helpful answer here, but before I managed to accept it was removed... Thank you anyway, and here it is for the reference:
https://hackage.haskell.org/package/generic-lens-2.0.0.0/docs/Data-Generics-Product-Fields.html
In short, the goal of testing failing types can be achieved with doctest, in the way the linked library does it.
I am following this tutorial on named tuple with specification of variable types. However, I modified the code (below), and even if I enter values of wrong types, there was no error message or programming break as a result. I understand you can write your own try/except to raise error exception, but is there a readily-available solution/syntax to enforce users entering the right type of variables.
from typing import NamedTuple
class Pet(NamedTuple):
pet_name: str
pet_type: str
def __repr__(self):
return f"{self.pet_name}, {self.pet_type}"
cleons_pet = Pet('Cotton', 'owl')
print('cleons_pet: ', cleons_pet)
cleons_pet_v2 = Pet(222, 1)
print('cleons_pet_v2: ', cleons_pet_v2)
# Output
cleons_pet: Cotton, owl
cleons_pet_v2: 222, 1
[Finished in 0.1s]
The type hints in python will not be evaluated by python itself! See PEP484
While these annotations are available at runtime through the usual annotations attribute, no type checking happens at runtime. Instead, the proposal assumes the existence of a separate off-line type checker which users can run over their source code voluntarily.
There are at least two projects which offer offline type checking (mypy and pyre). You should definitely use them if you are using type hints in your project.
If you want to validate the input while running the application, you have to either convince the offline type checkers by validating the data by yourself or use a third-party library. I know of attrs, where you can use validators or type annotations for online validation.
I ran into an issue when using mypy and have been able to find any help/reports regarding it. The following simplified code and error message should be self-explanatory:
#!/usr/bin/env python3
from typing import List, Union
class Corpus:
path: List[str]
def __init__(self, path:Union[str,List[str]]) -> None:
if type(path) == str:
self.path = [path]
else:
self.path = path
Mypy gives the following errors:
simplified.py:10: error: List item 0 has incompatible type "Union[str, List[str]]"; expected "str"
simplified.py:12: error: Incompatible types in assignment (expression has type "Union[str, List[str]]", variable has type "List[str]")
Even though the type of path variable is checked so that self.path should always result in string list, mypy complains about incompatible types.
Am I overlooking something or is this a bug in mypy?
(It it is a bug, should I go with #type: ignore annotation or is there a better work-around?)
(Some background: I decided to ease my life by writing a module which would take care of some repeating work. The argument in question should be a path to text data and I expect it to be only one string most of the time so I don't want to force putting it in a list. However, I wish to allow specifying more paths too. Internally, I store it as a list anyway as iterator over the class is always initialized with such list (and then possibly extends it further by "unpacking" directories)).
Try using isinstance(path, str) instead of type(path) == str. The former makes mypy typecheck your code without reporting an error.
Mypy really ought to support the latter form though -- there's an open feature request about it. The reason why this isn't implemented yet is almost certainly due to lack of time -- mypy's core team is pretty small, and there's an easy workaround in this case, so the feature was deprioritized.
(But hey, mypy is open source, so if you have some spare time...)