I have been after a way to provide none initialized instance variables to my class. I found that we can actually do that using type hinting without assigning anything to them. Which does not seem to create it in anyway. For example:
class T:
def __init__(self):
self.a: str
def just_print(self):
print(self.a)
def assign(self):
self.a = "test"
Now lets say I run this code:
t = T()
t.just_print()
It will raise an AttributeError saying 'T' object has not attribute 'a'. Obviously, when I run this code, it prints test.
t = T()
t.assign()
t.just_print()
My question is, what happens behind the scene when I just do a: str? It doesn't get added to the class's attributes. But it doesn't cause any problem either. So... is it just ignored? This is python 3.8 by the way.
You're referring to type annotations, as defined by PEP 526:
my_var: int
Please note that type annotations differ from type hints, as defined by PEP 428:
def my_func(foo: str):
...
Type annotations have actual runtime effects. For example, the documentation states:
In addition, at the module or class level, if the item being annotated is a simple name, then it and the annotation will be stored in the __annotations__ attribute of that module or class [...]
So, by slightly modifying your example, we get this:
>>> class T:
... a: str
...
>>> T.__annotations__
{'a': <class 'str'>}
Related
I have a class decorator, which adds a few functions and fields to decorated class.
#mydecorator
#dataclass
class A:
a: str = ""
Added (via setattr()) is a .save() function and a set of info for dataclass fields as a separate dict.
I'd like VScode and mypy to properly recognize that, so that when I use:
a=A()
a.save()
or a.my_fields_dict those 2 are properly recognized.
Is there any way to do that? Maybe modify class A type annotations at runtime?
TL;DR
What you are trying to do is not possible with the current type system.
1. Intersection types
If the attributes and methods you are adding to the class via your decorator are static (in the sense that they are not just known at runtime), then what you are describing is effectively the extension of any given class T by mixing in a protocol P. That protocol defines the method save and so on.
To annotate this you would need an intersection of T & P. It would look something like this:
from typing import Protocol, TypeVar
T = TypeVar("T")
class P(Protocol):
#staticmethod
def bar() -> str: ...
def dec(cls: type[T]) -> type[Intersection[T, P]]:
setattr(cls, "bar", lambda: "x")
return cls # type: ignore[return-value]
#dec
class A:
#staticmethod
def foo() -> int:
return 1
You might notice that the import of Intersection is conspicuously missing. That is because despite being one of the most requested features for the Python type system, it is still missing as of today. There is currently no way to express this concept in Python typing.
2. Class decorator problems
The only workaround right now is a custom implementation alongside a corresponding plugin for the type checker(s) of your choice. I just stumbled across the typing-protocol-intersection package, which does just that for mypy.
If you install that and add plugins = typing_protocol_intersection.mypy_plugin to your mypy configuration, you could write your code like this:
from typing import Protocol, TypeVar
from typing_protocol_intersection import ProtocolIntersection
T = TypeVar("T")
class P(Protocol):
#staticmethod
def bar() -> str: ...
def dec(cls: type[T]) -> type[ProtocolIntersection[T, P]]:
setattr(cls, "bar", lambda: "x")
return cls # type: ignore[return-value]
#dec
class A:
#staticmethod
def foo() -> int:
return 1
But here we run into the next problem. Testing this with reveal_type(A.bar()) via mypy will yield the following:
error: "Type[A]" has no attribute "bar" [attr-defined]
note: Revealed type is "Any"
Yet if we do this instead:
class A:
#staticmethod
def foo() -> int:
return 1
B = dec(A)
reveal_type(B.bar())
we get no complaints from mypy and note: Revealed type is "builtins.str". Even though what we did before was equivalent!
This is not a bug of the plugin, but of the mypy internals. It is another long-standing issue, that mypy does not handle class decorators correctly.
A person in that issue thread even mentioned your use case in conjunction with the desired intersection type.
DIY
In other words, you'll just have to wait until those two holes are patched. Or you can hope that at least the decorator issue by mypy is fixed soon-ish and write your own VSCode plugin for intersection types in the meantime. Maybe you can get together with the person behind that mypy plugin I mentioned above.
I'm trying to add mypy to my python project but I have found a roadblock. Let's say I have the following inheritance:
class BaseClass:
base_attribute: str
class A(BaseClass):
attribute_for_class_A: str
class B(BaseClass):
attribute_for_class_B: str
Now let's create some code that handle both instances of these classes, but without really knowing it:
#dataclass
class ClassUsingTheOthers:
fields: Dict[str, BaseClass]
def get_field(self, field_name: str) -> BaseClass:
field = self.fields.get(field_name)
if not field:
raise ValueError('Not found')
return field
The important bit here is the get_field method. Now let's create a function to use the get_field method, but that function will require to use a particlar subclass of BaseClass, B, for instance:
def function_that_needs_an_instance_of_b(instance: B):
print(instance.attribute_for_class_B)
Now if we use all the code together, we can get the following:
if __name__ == "__main__":
class_using_the_others = ClassUsingTheOthers(
fields={
'name_1': A(),
'name_2': B()
}
)
function_that_needs_an_instance_of_b(class_using_the_others.get_field('name_2'))
Obviously, when I run mypy to this file (in this gist you find all the code), I get the following error, as expected:
error: Argument 1 to "function_that_needs_an_instance_of_b" has incompatible type "BaseClass"; expected "B" [arg-type]
So my question is, how do I fix my code to make this error go away? I cannot change the type hint of the fields attribute because I really need to set it that way. Any ideas? Am I missing something? Should I check the type of the field returned?
I cannot change the type hint of the fields attribute
Well, there is your answer. If you declare fields to be a dictionary with the values type BaseClass, how do you expect any static type checker to know more about it?
(Related: Type annotation for mutable dictionary)
The type checker does not distinguish between different values of the dictionary based on any key you provide.
If you knew ahead of time, what the exact key-value-pairs can be, you could either do this with a TypedDict (as #dROOOze suggested) or you could write some ugly overloads with different Literal string values for field_name of your get_field method.
But none of those apply due to your restriction.
So you are left with either type-narrowing with a runtime assertion (as alluded to by #juanpa.arrivillaga), which I would recommend, or placing a specific type: ignore[arg-type] comment (as mentioned by #luk2302) and be done with it.
The former would look like this:
from dataclasses import dataclass
class BaseClass:
base_attribute: str
#dataclass
class A(BaseClass):
attribute_for_class_A: str
#dataclass
class B(BaseClass):
attribute_for_class_B: str
#dataclass
class ClassUsingTheOthers:
fields: dict[str, BaseClass]
def get_field(self, field_name: str) -> BaseClass:
field = self.fields.get(field_name)
if not field:
raise ValueError('Not found')
return field
def function_that_needs_an_instance_of_b(instance: B) -> None:
print(instance.attribute_for_class_B)
if __name__ == '__main__':
class_using_the_others = ClassUsingTheOthers(
fields={
'name_1': A(attribute_for_class_A='foo'),
'name_2': B(attribute_for_class_B='bar'),
}
)
obj = class_using_the_others.get_field('name_2')
assert isinstance(obj, B)
function_that_needs_an_instance_of_b(obj)
This both keeps mypy happy and you sane, if you ever forget, what value you were expecting there.
I'd like to type-annotate abstract class method witch behave as a constructor. For example in the code below, ElementBase.from_data is meant to be a abstract classmethod constructor.
tmp.py
from abc import abstractmethod, abstractclassmethod
import copy
from typing import TypeVar, Type
ElementT = TypeVar('ElementT', bound='ElementBase')
class ElementBase:
data: int
def __init__(self, data): self.data
##abstractmethod
def get_plus_one(self: ElementT) -> ElementT:
out = copy.deepcopy(self)
out.data = self.data + 1
return out
#abstractclassmethod
def from_data(cls: Type[ElementT], data: int) -> ElementT: # mypy error!!!
pass
class Concrete(ElementBase):
#classmethod
def from_data(cls, data: int) -> 'Concrete': # mypy error!!!
return cls(data)
However, applying mypy to this code shows the following erros.
tmp.py:18: error: The erased type of self "Type[tmp.ElementBase]" is not a supertype of its class "tmp.ElementBase"
tmp.py:23: error: Return type "Concrete" of "from_data" incompatible with return type <nothing> in supertype "ElementBase"
Do you have any idea to fix this error? Also, I'm specifically confused that the part of get_plus_one does not cause error, while only the part of abstractclassmethod does cause the error.
FYI, I want to make the abstract method constructor generic becaues I want to statically ensure that all subclass of ElementBase returns object with it's type when calling from_data.
[EDIT] comment out abstractmethod
It looks like mypy doesn't understand the abstractclassmethod decorator. That decorator has been deprecated since Python 3.3, as the abstractmethod and classmethod decorators were updated to play nice together. I think your code will work properly if you do:
#classmethod
#abstractmethod
def from_data(cls: Type[ElementT], data: int) -> ElementT:
pass
It's unrelated to your type checking issues, but you probably also want to change ElementBase to inherit from abc.ABC or to explicitly request the abc.ABCMeta metaclass if you want the abstractness of the class to be enforced by Python. Regular classes don't care about the abstractmethod decorator, and so as written, you'll be able to instantiate ElementBase (or you could if it's __init__ method didn't have an unrelated issue).
And another peripherally related note on this kind of type hinting... PEP 673 will add typing.Self in Python 3.11, which will be a convenient way for a method to refer to the type of object it's being called on. It should play nicely with classmethods without requiring you to jump through any hoops. With it you'd be able to write this much simpler version of the annotations:
#classmethod
#abstractmethod
def from_data(cls, data: int) -> Self:
pass
I want to get the type hints for an object's attributes. I can only get the hints for the class and not an instance of it.
I have tried using foo_instance.__class__ from here but that only shows the class variables.
So in the example how do I get the type hint of bar?
class foo:
var: int = 42
def __init__(self):
self.bar: int = 2
print(get_type_hints(foo)) # returns {'var': <class 'int'>}
I just had the same problem. The python doc isn't that clear since the example is made with what is now officially called dataclass.
Student(NamedTuple):
name: Annotated[str, 'some marker']
get_type_hints(Student) == {'name': str}
get_type_hints(Student, include_extras=False) == {'name': str}
get_type_hints(Student, include_extras=True) == {
'name': Annotated[str, 'some marker']
}
It give the impression that get_type_hints() works on class directly. Turns out get_type_hints() returns hints based on functions, not on class. That way it can be use with both if we know that. A normal class obviously not being instantiated at it's declaration, it does not have any of the variables set within the __init__() method who hasn't yet been called. It couldn't be that way either if we want the possibility to get the type hints from class-wide variables.
So you could either call it on __init__(), that is if variables are passed in arguments though (yes i seen it's not in your example but might help others since i didn't seen this anywhere in hours of search);
class foo:
var: int = 42
def __init__(self, bar: int = 2):
self.bar = int
print(get_type_hints(foo.__init__))
At last for your exact example i believe you have two choices. You could instantiate a temporary object and use del to clean it right after if your logic allows it. Or declare your variables as class ones with or without default values so you can get them with get_type_hints() and assign them later in instantiations.
Maybe this is a hack, and you have to be the creator of your instances, but there are a subset of cases in which using a data class will get you what you want;
Python 3.7+
#dataclass
class Foo:
bar: str = 2
if __name__ == '__main__':
f = Foo()
print(f.bar)
print(get_type_hints(f))
2
{'bar': <class 'str'>}
Hints only exist at the class level — by the time an instance is created the type of its attributes will be that of whatever value has been assigned to them. You can get the type of any instance attribute by using the first form of the built-in type() function — e.g. type(foo_instance.var).
This information isn't evaluated and only exists in the source code.
if you must get this information, you can use the ast module and extract the information from the source code yourself, if you have access to the source code.
You should also ask yourself if you need this information because in most cases reevaluating the source code will be to much effort.
How can I cast a var into a CustomClass?
In Python, I can use float(var), int(var) and str(var) to cast a variable into primitive data types but I can't use CustomClass(var) to cast a variable into a CustomClass unless I have a constructor for that variable type.
Example with inheritance.
class CustomBase:
pass
class CustomClass(CustomBase):
def foo():
pass
def bar(var: CustomBase):
if isinstance(var, CustomClass):
# customClass = CustomClass(var) <-- Would like to cast here...
# customClass.foo() <-- to make it clear that I can call foo here.
In the process of writing this question I believe I've found a solution.
Python is using Duck-typing
Therefore it is not necessary to cast before calling a function.
Ie. the following is functionally fine.
def bar(var):
if isinstance(var, CustomClass):
customClass.foo()
I actually wanted static type casting on variables
I want this so that I can continue to get all the lovely benefits of the typing PEP in my IDE such as checking function input types, warnings for non-existant class methods, autocompleting methods, etc.
For this I believe re-typing (not sure if this is the correct term) is a suitable solution:
class CustomBase:
pass
class CustomClass(CustomBase):
def foo():
pass
def bar(var: CustomBase):
if isinstance(var, CustomClass):
customClass: CustomClass = var
customClass.foo() # Now my IDE doesn't report this method call as a warning.