When i use the subclass method, a method that base class don't have, comes the warning.
Code is as following:
class A:
pass
class B(A):
def b_method(self):
pass
def test(arg: A):
arg.b_method() #generate warning at this line
warning:Unresolved attribute reference 'b_method' for class 'A'
The variables of the input function can only be class A and its subclasses, and the function will call the unique method of some subclasses. How to write type annotations to eliminate the warning?
Any help would be greatly appreciated!
The behavior you are seeing is expected.
A is the parent class
B is child class.
B can access a method that A has but not vice versa..
Makes sense?
Related
I have a mixin class WithWorkMixin that has one abstract method def do_work(self) -> None.
I also have a metaclass to enforce singleton of child classes. Following is its definition.
from abc import ABCMeta
from typing import Any
from typing import Dict
class Singleton(ABCMeta):
_instances: Dict[type, Any] = {}
def __call__(cls: Any, *args: Any, **kwargs: Any) -> Any:
if cls not in cls._instances:
cls._instances[cls] = super().__call__(*args, **kwargs)
return cls._instances[cls]
Now, when I want to create a class A that inherits WithWorkMixin and uses the Singleton metaclass at the same time like the following,
class A(WithWorkMixin, metaclass=Singleton):
the abstract method in WithWorkMixin doesn't seem to be enforced. I can create a new instance named a of A without defining do_work. When I do type checking on that new instance, it looks like a is an instance of WithWorkMixin.
isinstance(a, WithWorkMixin) # return True
Could someone help me understand what's going on? Is there anything I can do to make it work? I guess I probably need to specify the inheritance within the Singleton class somehow. Thanks.
P.S. I did some additional tests. If I directly add an abstractmethod in A, it does give the error message as expected.
I'd like to type-annotate abstract class method witch behave as a constructor. For example in the code below, ElementBase.from_data is meant to be a abstract classmethod constructor.
tmp.py
from abc import abstractmethod, abstractclassmethod
import copy
from typing import TypeVar, Type
ElementT = TypeVar('ElementT', bound='ElementBase')
class ElementBase:
data: int
def __init__(self, data): self.data
##abstractmethod
def get_plus_one(self: ElementT) -> ElementT:
out = copy.deepcopy(self)
out.data = self.data + 1
return out
#abstractclassmethod
def from_data(cls: Type[ElementT], data: int) -> ElementT: # mypy error!!!
pass
class Concrete(ElementBase):
#classmethod
def from_data(cls, data: int) -> 'Concrete': # mypy error!!!
return cls(data)
However, applying mypy to this code shows the following erros.
tmp.py:18: error: The erased type of self "Type[tmp.ElementBase]" is not a supertype of its class "tmp.ElementBase"
tmp.py:23: error: Return type "Concrete" of "from_data" incompatible with return type <nothing> in supertype "ElementBase"
Do you have any idea to fix this error? Also, I'm specifically confused that the part of get_plus_one does not cause error, while only the part of abstractclassmethod does cause the error.
FYI, I want to make the abstract method constructor generic becaues I want to statically ensure that all subclass of ElementBase returns object with it's type when calling from_data.
[EDIT] comment out abstractmethod
It looks like mypy doesn't understand the abstractclassmethod decorator. That decorator has been deprecated since Python 3.3, as the abstractmethod and classmethod decorators were updated to play nice together. I think your code will work properly if you do:
#classmethod
#abstractmethod
def from_data(cls: Type[ElementT], data: int) -> ElementT:
pass
It's unrelated to your type checking issues, but you probably also want to change ElementBase to inherit from abc.ABC or to explicitly request the abc.ABCMeta metaclass if you want the abstractness of the class to be enforced by Python. Regular classes don't care about the abstractmethod decorator, and so as written, you'll be able to instantiate ElementBase (or you could if it's __init__ method didn't have an unrelated issue).
And another peripherally related note on this kind of type hinting... PEP 673 will add typing.Self in Python 3.11, which will be a convenient way for a method to refer to the type of object it's being called on. It should play nicely with classmethods without requiring you to jump through any hoops. With it you'd be able to write this much simpler version of the annotations:
#classmethod
#abstractmethod
def from_data(cls, data: int) -> Self:
pass
1. A problem with dataclass mix-ins, solved
To make abstract dataclasses that type-check under mypy, I've been breaking them into two classes, one that contains the abstract methods and one that contains the data members, as explained in this answer. The abstract class inherits from the dataclass. This runs into a problem, though, when another abstract-class-and-dataclass pair inherits from the first one: the "ancestor" dataclass's fields get wiped out by the "descendant". For example:
from dataclasses import dataclass
from abc import ABC, abstractmethod
#dataclass
class ADataclassMixin:
a_field: int = 1
class A(ADataclassMixin, ABC):
#abstractmethod
def method(self):
pass
#dataclass
#class BDataclassMixin(A): # works but fails mypy 0.931 type-check
class BDataclassMixin: # fails
b_field: int = 2
pass
class B(BDataclassMixin, A):
def method(self):
return self
o = B(a_field=5)
The last line fails, yielding this error message:
TypeError: BDataclassMixin.__init__() got an unexpected keyword argument 'a_field'
B's method-resolution order (B.__mro__) is (B, BDataclassMixin, A, ADataclassMixin, ABC, object), as expected. But a_field is not found.
A solution, shown in the commented-out line above, is to put the ancestor class explicitly in the descendant dataclass's declaration: class BDataclassMixin(A) instead of class BDataclassMixin. This fails type-checking, though, because a dataclass can only be a concrete class.
2. A problem with that solution, unsolved
The above solution breaks down if we add a third class, inheriting from B:
#dataclass
#class CDataclassMixin: # fails
class CDataclassMixin(A): # fails
#class CDataclassMixin(B, A): # works but fails type-check
c_field: int = 3
pass
class C(CDataclassMixin, B):
def method(self):
return "C's result"
pass
o = C(b_field=5)
Now, C has a_field and c_field but has lost b_field.
I have found that if I declare CDataclassMixin explicitly to inherit from B and A (in that order), b_field will be in the resulting class along with a_field_ and c_field`. However, explicitly stating the inheritance hierarchy in every mix-in defeats the purpose of mix-ins, which is to be able to code them independently of all the other mix-ins and to mix them easily and any way you like.
What is the correct way to make abstract dataclass mix-ins, so that classes that inherit from them include all the dataclass fields?
The correct solution is to abandon the DataclassMixin classes and simply make the abstract classes into dataclasses, like this:
#dataclass # type: ignore[misc]
class A(ABC):
a_field: int = 1
#abstractmethod
def method(self):
pass
#dataclass # type: ignore[misc]
class B(A):
b_field: int = 2
#dataclass
class C(B):
c_field: int = 3
def method(self):
return self
The reason for the failures is that, as explained in the documentation on dataclasses, the complete set of fields in a dataclass is determined when the dataclass is compiled, not when it is inherited from. The internal code that generates the dataclass's __init__ function can only examine the MRO of the dataclass as it is declared on its own, not when mixed in to another class.
It's necessary to add # type: ignore[misc] to each abstract dataclass's #dataclass line, not because the solution is wrong but because mypy is wrong. It is mypy, not Python, that requires dataclasses to be concrete. As explained by ilevkivskyi in mypy issue 5374, the problem is that mypy wants a dataclass to be a Type object and for every Type object to be capable of being instantiated. This is a known problem and awaits a resolution.
The behavior in the question and in the solution is exactly how dataclasses should behave. And, happily, abstract dataclasses that inherit this way (the ordinary way) can be mixed into other classes willy-nilly no differently than other mix-ins.
Putting the mixin as the last base class works without error:
#dataclass
class ADataclassMixin:
a_field: int = 1
class A(ABC, ADataclassMixin):
#abstractmethod
def method(self):
pass
#dataclass
class BDataclassMixin:
b_field: int = 2
class B(A, BDataclassMixin):
def method(self):
return self
o = B(a_field=5)
print((o.a_field, o.b_field)) # (5,2)
My question was inspired by this question.
The problem there is the 3 level class model - the terminating classes (3-rd level) only should be stored in the registry, but the 2-nd level are interfering and also have stored, because they are subclasses of 1-st level.
I wanted to get rid of 1-st level class by using metaclass. By this way the only 2 class levels are left - base classes for each group of settings and their childs - various setting classes, inherited from the according base class. The metaclass serves as a class factory - it should create base classes with needed methods and shouldn't be displayed in the inheritance tree.
But my idea doesn't work, because it seems that the __init_subclass__ method (the link to method) doesn't copied from the metaclass to constructed classes. In contrast of __init__ method, that works as I were expected.
Code snippet № 1. The basic framework of the model:
class Meta_Parent(type):
pass
class Parent_One(metaclass=Meta_Parent):
pass
class Child_A(Parent_One):
pass
class Child_B(Parent_One):
pass
class Child_C(Parent_One):
pass
print(Parent_One.__subclasses__())
Output:
[<class '__main__.Child_A'>, <class '__main__.Child_B'>, <class '__main__.Child_C'>]
I have wanted to add functionality to the subclassing process of the above model, so I have redefined the type's builtin __init_subclass__ like this:
Code snippet № 2.
class Meta_Parent(type):
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
print(cls)
From my point of view, now every new class, constructed by Meta_Parent metaclass (for example, Parent_One) should have __init_subclass__ method and thus, should print the subclass name when every class is inherited from this new class, but it prints nothing. That is, my __init_subclass__ method doesn't called when inheritance happens.
It works if Meta_Parent metaclass is directly inherited though:
Code snippet № 3.
class Meta_Parent(type):
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
print(cls)
class Child_A(Meta_Parent):
pass
class Child_B(Meta_Parent):
pass
class Child_C(Meta_Parent):
pass
Output:
<class '__main__.Child_A'>
<class '__main__.Child_B'>
<class '__main__.Child_C'>
Nothing strange here, the __init_subclass__ was created exactly for this purpose.
I were thinking at a moment, that dunder methods are belong to metaclass only and are not passed to new constructed classes, but then, I try the __init__ method and it works as I were expecting in the beginning - looks like the link to __init__ have copied to every metaclass's class.
Code snippet № 4.
class Meta_Parent(type):
def __init__(cls, name, base, dct):
super().__init__(name, base, dct)
print(cls)
Output:
<class '__main__.Parent_One'>
<class '__main__.Child_A'>
<class '__main__.Child_B'>
<class '__main__.Child_C'>
The questions:
Why __init__ works, but __init_subclass__ doesn't?
Is it possible to implement my idea by using metaclass?
1. Why __init__ works, but __init_subclass__ doesn't?
I found the answer by debugging CPython by GDB.
The creation of a new class (type) starts in the type_call() function. It does two main things: a new type object creation and this object initialization.
obj = type->tp_new(type, args, kwds); is an object creation. It calls the type's tp_new slot with passed arguments. By default the tp_new stores reference to the basic type object's tp_new slot, but if any ancestor class implements the __new__ method, the reference is changing to the slot_tp_new dispatcher function. Then the type->tp_new(type, args, kwds); callsslot_tp_new function and it, in own turn, invokes the search of __new__ method in the mro chain. The same happens with tp_init.
The subclass initialization happens at the end of new type creation - init_subclass(type, kwds). It searches the __init_subclass__ method in the mro chain of the just created new object by using the super object. In my case the object's mro chain has two items:
print(Parent_One.__mro__)
### Output
(<class '__main__.Parent_One'>, <class 'object'>).
int res = type->tp_init(obj, args, kwds); is an object initialization. It also searches the __init__ method in the mro chain, but use the metaclass mro, not the just created new object's mro. In my case the metaclass mro has three item:
print(Meta_Parent.__mro__)
###Output
(<class '__main__.Meta_Parent'>, <class 'type'>, <class 'object'>)
The simplified execution diagram:
So, the answer is: __init_subclass__ and __init__ methods are searched in the different places:
the __init_subclass__ firstly is searched in the Parent_One's __dict__, then in the object's __dict__.
the __init__ is searched in this order: Meta_Parent's __dict__, type's __dict__, object's __dict__.
2. Is it possible to implement my idea by using metaclass?
I came up with following solution. It has drawback - the __init__ method is called by each subclass, the children included, that means - all subclasses have registry and __init_subclass__ attributes, which is needless. But it works as I were requesting in the question.
#!/usr/bin/python3
class Meta_Parent(type):
def __init__(cls, name, base, dct, **kwargs):
super().__init__(name, base, dct)
# Add the registry attribute to the each new child class.
# It is not needed in the terminal children though.
cls.registry = {}
#classmethod
def __init_subclass__(cls, setting=None, **kwargs):
super().__init_subclass__(**kwargs)
cls.registry[setting] = cls
# Assign the nested classmethod to the "__init_subclass__" attribute
# of each child class.
# It isn't needed in the terminal children too.
# May be there is a way to avoid adding these needless attributes
# (registry, __init_subclass__) to there. I don't think about it yet.
cls.__init_subclass__ = __init_subclass__
# Create two base classes.
# All child subclasses will be inherited from them.
class Parent_One(metaclass=Meta_Parent):
pass
class Parent_Two(metaclass=Meta_Parent):
pass
### Parent_One's childs
class Child_A(Parent_One, setting='Child_A'):
pass
class Child_B(Parent_One, setting='Child_B'):
pass
class Child_C(Parent_One, setting='Child_C'):
pass
### Parent_Two's childs
class Child_E(Parent_Two, setting='Child_E'):
pass
class Child_D(Parent_Two, setting='Child_D'):
pass
# Print results.
print("Parent_One.registry: ", Parent_One.registry)
print("#" * 100, "\n")
print("Parent_Two.registry: ", Parent_Two.registry)
Output
Parent_One.registry: {'Child_A': <class '__main__.Child_A'>, 'Child_B': <class '__main__.Child_B'>, 'Child_C': <class '__main__.Child_C'>}
####################################################################################################
Parent_Two.registry: {'Child_E': <class '__main__.Child_E'>, 'Child_D': <class '__main__.Child_D'>}
The solution I came up with and use/like is:
class Meta_Parent(type):
def _init_subclass_override(cls, **kwargs):
super().__init_subclass__(**kwargs)
# Do whatever... I raise an exception if something is wrong
#
# i.e
# if sub-class's name does not start with "Child_"
# raise NameError
#
# cls is the actual class, Child_A in this case
class Parent_One(metaclass=Meta_Parent):
#classmethod
def __init_subclass__(cls, **kwargs):
Meta_Parent._init_subclass_override(cls, **kwargs)
### Parent_One's childs
class Child_A(Parent_One):
pass
I like this because it DRYs the sub-class creation code/checks. At the same time, if you see Parent_One, you know that there is something happening whenever a sub-class is created.
I did it while mucking around to mimic my own Interface functionality (instead of using ABC), and the override method checks for existence of certain methods in the sub-classes.
One can argue whether the override method really belongs in the metaclass, or somewhere else.
In one of my classes, I am printing data from another class that is yet to be initialized.I only want to print that data once the class has been initialized.Is there any way check if the class has been instantiated?
Two functions that return true if you pass an undeclared class, and false if it is instantiated:
import inspect
inspect.isclass(myclass)
or
isinstance(myclass, type)
In general, if it's not a type (i.e. undeclared class), it's an instantiated type.
Simply add a variable into the class to be made, like this:
class tobeinitiated():
initiated=False
def __init__(self):
global initiated
tobeinitiated.initiated = True
Then, where you need the information:
global initiated #(if in class)
if tobeinitiated.initiated:
#do the stuff you need to do
Hope this helps. :)
You can add a counter of instances in the constructor for example.