Set variable as type of class - python-3.x

I am trying to figure out how I can pass a variable as the declaration type (object) for a class in Python 3.
Example:
#class defintion
class TestClass(Document):
test = IntField()
me = MongoEngine(app)
testInstance = TestClass(me.Document) # How do i pass the Document variable
I tried passing an instance of the MongoEngine variable as a variable to the TestClass but this isn't working properly?

I think you need to structure your class slightly different. Don't put Document in the class definition as if the TestClass is a subclass of Document. In stead, declare the class as standard (object), and define an __init__ where you can pass a variable which can be used by the instance of the class after initiation:
class TestClass(object):
def __init__(self, my_document):
self.document = my_document
# at this point the self.document variable
# is the same as the variable passed
# when initiating the instance of the class
def show_document(self):
# do something with your document
print(self.document)
me = MongoEngine(app)
# this will call __init__() passing the variable
test_instance = TestClass(me.Document)
# now do something with the class intance
test_instance.show_document()
[EDIT based on comment]
OP's comment:
Looking at the type(test_instance), Its not the same as a
MongoEngine.Document. I am hoping to create a class of type 'Document'
and pass in an instance of that type?
You can create classes which would take a parent class as object in the class definition. As I do not know MongoEngine I will make an example with list
A class defined as follows, will behave perfectly like a list, but if you do a type() it will come back as MyList:
class MyList(list):
def __init__(self, *args, **kwargs):
super(MyList, self).__init__(*args, **kwargs)
def my_extra_function(self):
print('hello world')
You can easily see this when using this class, first look at it as a list:
my_instance = MyList([1, 2, 3])
print(my_instance)
print(my_instance[::-1])
this will behave as if it was a list.
But when you do a type(), it will not return the same as list:
print(type(list))
print(type(list()))
print(type(MyList()))
print(type(my_instance))
output:
<class 'type'>
<class 'list'>
<class '__main__.MyList'>
<class '__main__.MyList'>
So even when you try to create a class with the MongoEngine.Document as parent object, the type() will still show you your own defined class.
class MyClass(MongoEngine.Document):
def __init__(self, *args, **kwargs):
super(MyClass, self).__init__(*args, **kwargs)
my_instance = MyClass('something')
If you do a type(my_instance) it will return your custom class, and not the parent object type.
Not sure how MongoEngine works, and if you can actually do something like this, so YMMV.
You can change the name type() is returning, by doing the following in my example class. Setting the self.__class__ in the __init__(). Like this:
class MyList(list):
def __init__(self, *args, **kwargs):
super(MyList, self).__init__(*args, **kwargs)
self.__class__ = type('list', (list,),{})
def my_extra_function(self):
print('hello world', self)
my_instance = MyList([1, 2, 3])
print(type(list))
print(type(list()))
print(type(MyList()))
print(type(my_instance))
output:
<class 'type'>
<class 'list'>
<class '__main__.list'>
<class '__main__.list'>
If this trick works for MongoEngine.Document I do not know.

Related

refering to instance of a class from its metaclass python

Is there any way to refer to instance of a class from its metaclass every time an instance is created? I suppose I should use dunder _call_ method inside metaclass for that purpose.
I have the following code:
class meta(type):
def __call__(cls):
super().__call__()
#<--- want to get an object of A class here every time when instance of A class is created
class A(metaclass = meta):
def __init__(self, c):
self.c = 2
def test(self):
print('test called')
a1=A()
a2=A()
a3=A()
Also why when I implement __call__ method inside metaclass all created instances of my class became NoneType however when overring __call__ I used super().__call__()?
For example a4.test() returns AttributeError: 'NoneType' object has no attribute 'test'
The newly created instance is returned by super().__call__() - you hav to keep this value in a variable, use t for whatever you want and return it.
Otherwise, if the metaclass __call__ has no return statement, all instances are imediatelly de-referenced and destroyed, and the code trying to create instances just get None:
class meta(type):
def __call__(cls):
obj = super().__call__()
# use obj as you see fit
...
return obj

Better way than pass the same argument among classes in Python

I have a question related to OOP but it should be implemented in Python.
I have a file user_inputs.py with all the user parameters.
In my main file, I have a function that is called first. This function is responsible to read all the user parameters and return a dictionary that will be used in the rest of the program.
My question is: what is the cleanest way to pass the user_parameters dictionary to all classes? I did it in 2 ways:
Method 1)
def read_user_parameters():
# code to open and read all parameters etc.
return user_parameters # returns a dictionary with all the user parameters
Class A():
def __init__(self, user_parameters):
self.user_parameters = user_parameters
Class B():
def __init__(self, user_parameters):
self.user_parameters = user_parameters
user_parameters = read_user_parameters()
object_A = A(user_parameters)
object_B = B(user_parameters)
I don't like this way because I have dozens of classes that need to pass this argument. So I thought to create a parent class with the user parameters:
Method 2)
Class User_parameters():
def __init__(self, user_parameters):
def read_user_parameters():
# code to open and read all parameters etc.
return user_parameters
Class A(User_parameters):
__init__(self, user_parameters):
super().__init__()
# self.user_parameters comes from the parent class now
Class B(User_parameters):
__init__(self, user_parameters):
super().__init__()
# self.user_parameters comes from the parent class now
object_A = A()
object_B = B()
I prefer method 2, however, when super() is initialized from Class A and Class B the function read_user_parameters() that reads the file will be called twice (multiply this by dozens of times). Is there a better solution than method 1 in which I call read_user_parameters() only once but doesn't need to pass the argument for all classes?
Thank you for your time.
Why not just have a single UserParameters class and two objects of the same class (Also class nameds are supposed to be camel-cases, not snake-cased)
#Single class for user parameters
class UserParameters:
def __init__(self, user_parameters):
self.user_parameters = user_parameters
def read_user_parameters(self):
# code to open and read all parameters etc.
return self.user_parameters
#Two objects
object_A = UserParameters("<params>")
object_B = UserParameters("<params>")

Python subclass that takes superclass as argument on instantiation?

I am trying to create a wrapper class in Python with the following behaviour:
It should take as an argument an existing class from which it should inherit all methods and attributes
The wrapper class methods should be able to use Python super() to access methods of the superclass (the one passed as an argument)
Because of my second requirement I think the solution here will not suffice (and in any case I am having separate issues deepcopying some of the methods of the superclass' I am trying to inherit from).
I tried this but it's not correct...
class A:
def shout(self):
print("I AM A!")
class B:
def shout(self):
print("My name is B!")
class wrapper:
def __init__(self, super_class):
## Some inheritance thing here ##
# I initially tried this but no success...
super(super_class).__init__() # or similar?
def shout(self):
print('This is a wrapper')
super().shout()
And this is the behaviour I require...
my_wrapper = wrapper(A)
my_wrapper.shout()
# Expected output:
# > This is a wrapper
# > I AM A
my_wrapper = wrapper(B)
my_wrapper.shout()
# Expected output:
# > This is a wrapper
# > My name is B!
Is inheritance the correct approach here, if so am I sniffing in the right direction? Any help is appreciated, thanks :)
Edit for context:
I intend to build multiple wrappers so that all of my ML models have the same API. Generally, models from the same package (sklearn for example) have the same API and should be able to be wrapped by the same wrapper. In doing this I wish to modify/add functionality to the existing methods in these models whilst keeping the same method name.
If wrapper has to be a class then a composition solution would fit much better here.
Keep in mind that I turned the shout methods to staticmethod because in your example you pass the class to wrapper.shout, not an instance.
class A:
#staticmethod
def shout():
print("I AM A!")
class B:
#staticmethod
def shout():
print("My name is B!")
class wrapper:
def __init__(self, super_class):
self._super_class = super_class
def __getattr__(self, item):
try:
return self.__dict__[item].__func__
except KeyError:
return self._super_class.__dict__[item].__func__
def a_wrapper_method(self):
print('a wrapper attribute can still be used')
my_wrapper = wrapper(A)
my_wrapper.shout()
my_wrapper = wrapper(B)
my_wrapper.shout()
my_wrapper.a_wrapper_method()
Outputs
This is a wrapper
I AM A!
This is a wrapper
My name is B!
a wrapper attribute can still be used
So I went for a function in the end. My final solution:
class A:
def shout(self):
print("I AM A!")
class B:
def shout(self):
print("My name is B!")
def wrap_letter_class(to_wrap):
global letterWrapper
class letterWrapper(to_wrap):
def __init__(self):
super().__init__()
def shout(self):
print('This is a wrapper')
super().shout()
def __getstate__(self):
# Add the wrapper to global scope before pickling
global letterWrapper
letterWrapper = self.__class__
return self.__dict__
return letterWrapper()
Which produces the desired behaviour...
In [2]: wrapped = wrap_letter_class(A)
In [3]: wrapped.shout()
This is a wrapper
I AM A!
In [4]: wrapped = wrap_letter_class(B)
In [5]: wrapped.shout()
This is a wrapper
My name is B!
Something not mentioned in my initial question was that I intended to pickle my custom class, this is not possible if the class is not defined in the global scope, hence the __getstate__ and global additions.
Thanks!

Python Is it ok that an attribute only exists in child/concrete classes [duplicate]

What's the best practice to define an abstract instance attribute, but not as a property?
I would like to write something like:
class AbstractFoo(metaclass=ABCMeta):
#property
#abstractmethod
def bar(self):
pass
class Foo(AbstractFoo):
def __init__(self):
self.bar = 3
Instead of:
class Foo(AbstractFoo):
def __init__(self):
self._bar = 3
#property
def bar(self):
return self._bar
#bar.setter
def setbar(self, bar):
self._bar = bar
#bar.deleter
def delbar(self):
del self._bar
Properties are handy, but for simple attribute requiring no computation they are an overkill. This is especially important for abstract classes which will be subclassed and implemented by the user (I don't want to force someone to use #property when he just could have written self.foo = foo in the __init__).
Abstract attributes in Python question proposes as only answer to use #property and #abstractmethod: it doesn't answer my question.
The ActiveState recipe for an abstract class attribute via AbstractAttribute may be the right way, but I am not sure. It also only works with class attributes and not instance attributes.
A possibly a bit better solution compared to the accepted answer:
from better_abc import ABCMeta, abstract_attribute # see below
class AbstractFoo(metaclass=ABCMeta):
#abstract_attribute
def bar(self):
pass
class Foo(AbstractFoo):
def __init__(self):
self.bar = 3
class BadFoo(AbstractFoo):
def __init__(self):
pass
It will behave like this:
Foo() # ok
BadFoo() # will raise: NotImplementedError: Can't instantiate abstract class BadFoo
# with abstract attributes: bar
This answer uses same approach as the accepted answer, but integrates well with built-in ABC and does not require boilerplate of check_bar() helpers.
Here is the better_abc.py content:
from abc import ABCMeta as NativeABCMeta
class DummyAttribute:
pass
def abstract_attribute(obj=None):
if obj is None:
obj = DummyAttribute()
obj.__is_abstract_attribute__ = True
return obj
class ABCMeta(NativeABCMeta):
def __call__(cls, *args, **kwargs):
instance = NativeABCMeta.__call__(cls, *args, **kwargs)
abstract_attributes = {
name
for name in dir(instance)
if getattr(getattr(instance, name), '__is_abstract_attribute__', False)
}
if abstract_attributes:
raise NotImplementedError(
"Can't instantiate abstract class {} with"
" abstract attributes: {}".format(
cls.__name__,
', '.join(abstract_attributes)
)
)
return instance
The nice thing is that you can do:
class AbstractFoo(metaclass=ABCMeta):
bar = abstract_attribute()
and it will work same as above.
Also one can use:
class ABC(ABCMeta):
pass
to define custom ABC helper. PS. I consider this code to be CC0.
This could be improved by using AST parser to raise earlier (on class declaration) by scanning the __init__ code, but it seems to be an overkill for now (unless someone is willing to implement).
2021: typing support
You can use:
from typing import cast, Any, Callable, TypeVar
R = TypeVar('R')
def abstract_attribute(obj: Callable[[Any], R] = None) -> R:
_obj = cast(Any, obj)
if obj is None:
_obj = DummyAttribute()
_obj.__is_abstract_attribute__ = True
return cast(R, _obj)
which will let mypy highlight some typing issues
class AbstractFooTyped(metaclass=ABCMeta):
#abstract_attribute
def bar(self) -> int:
pass
class FooTyped(AbstractFooTyped):
def __init__(self):
# skipping assignment (which is required!) to demonstrate
# that it works independent of when the assignment is made
pass
f_typed = FooTyped()
_ = f_typed.bar + 'test' # Mypy: Unsupported operand types for + ("int" and "str")
FooTyped.bar = 'test' # Mypy: Incompatible types in assignment (expression has type "str", variable has type "int")
FooTyped.bar + 'test' # Mypy: Unsupported operand types for + ("int" and "str")
and for the shorthand notation, as suggested by #SMiller in the comments:
class AbstractFooTypedShorthand(metaclass=ABCMeta):
bar: int = abstract_attribute()
AbstractFooTypedShorthand.bar += 'test' # Mypy: Unsupported operand types for + ("int" and "str")
Just because you define it as an abstractproperty on the abstract base class doesn't mean you have to make a property on the subclass.
e.g. you can:
In [1]: from abc import ABCMeta, abstractproperty
In [2]: class X(metaclass=ABCMeta):
...: #abstractproperty
...: def required(self):
...: raise NotImplementedError
...:
In [3]: class Y(X):
...: required = True
...:
In [4]: Y()
Out[4]: <__main__.Y at 0x10ae0d390>
If you want to initialise the value in __init__ you can do this:
In [5]: class Z(X):
...: required = None
...: def __init__(self, value):
...: self.required = value
...:
In [6]: Z(value=3)
Out[6]: <__main__.Z at 0x10ae15a20>
Since Python 3.3 abstractproperty is deprecated. So Python 3 users should use the following instead:
from abc import ABCMeta, abstractmethod
class X(metaclass=ABCMeta):
#property
#abstractmethod
def required(self):
raise NotImplementedError
If you really want to enforce that a subclass define a given attribute, you can use metaclasses:
class AbstractFooMeta(type):
def __call__(cls, *args, **kwargs):
"""Called when you call Foo(*args, **kwargs) """
obj = type.__call__(cls, *args, **kwargs)
obj.check_bar()
return obj
class AbstractFoo(object):
__metaclass__ = AbstractFooMeta
bar = None
def check_bar(self):
if self.bar is None:
raise NotImplementedError('Subclasses must define bar')
class GoodFoo(AbstractFoo):
def __init__(self):
self.bar = 3
class BadFoo(AbstractFoo):
def __init__(self):
pass
Basically the meta class redefine __call__ to make sure check_bar is called after the init on an instance.
GoodFoo()  # ok
BadFoo ()  # yield NotImplementedError
As Anentropic said, you don't have to implement an abstractproperty as another property.
However, one thing all answers seem to neglect is Python's member slots (the __slots__ class attribute). Users of your ABCs required to implement abstract properties could simply define them within __slots__ if all that's needed is a data attribute.
So with something like,
class AbstractFoo(abc.ABC):
__slots__ = ()
bar = abc.abstractproperty()
Users can define sub-classes simply like,
class Foo(AbstractFoo):
__slots__ = 'bar', # the only requirement
# define Foo as desired
def __init__(self):
self.bar = ...
Here, Foo.bar behaves like a regular instance attribute, which it is, just implemented differently. This is simple, efficient, and avoids the #property boilerplate that you described.
This works whether or not ABCs define __slots__ at their class' bodies. However, going with __slots__ all the way not only saves memory and provides faster attribute accesses but also gives a meaningful descriptor instead of having intermediates (e.g. bar = None or similar) in sub-classes.1
A few answers suggest doing the "abstract" attribute check after instantiation (i.e. at the meta-class __call__() method) but I find that not only wasteful but also potentially inefficient as the initialization step could be a time-consuming one.
In short, what's required for sub-classes of ABCs is to override the relevant descriptor (be it a property or a method), it doesn't matter how, and documenting to your users that it's possible to use __slots__ as implementation for abstract properties seems to me as the more adequate approach.
1 In any case, at the very least, ABCs should always define an empty __slots__ class attribute because otherwise sub-classes are forced to have __dict__ (dynamic attribute access) and __weakref__ (weak reference support) when instantiated. See the abc or collections.abc modules for examples of this being the case within the standard library.
The problem isn't what, but when:
from abc import ABCMeta, abstractmethod
class AbstractFoo(metaclass=ABCMeta):
#abstractmethod
def bar():
pass
class Foo(AbstractFoo):
bar = object()
isinstance(Foo(), AbstractFoo)
#>>> True
It doesn't matter that bar isn't a method! The problem is that __subclasshook__, the method of doing the check, is a classmethod, so only cares whether the class, not the instance, has the attribute.
I suggest you just don't force this, as it's a hard problem. The alternative is forcing them to predefine the attribute, but that just leaves around dummy attributes that just silence errors.
I've searched around for this for awhile but didn't see anything I like. As you probably know if you do:
class AbstractFoo(object):
#property
def bar(self):
raise NotImplementedError(
"Subclasses of AbstractFoo must set an instance attribute "
"self._bar in it's __init__ method")
class Foo(AbstractFoo):
def __init__(self):
self.bar = "bar"
f = Foo()
You get an AttributeError: can't set attribute which is annoying.
To get around this you can do:
class AbstractFoo(object):
#property
def bar(self):
try:
return self._bar
except AttributeError:
raise NotImplementedError(
"Subclasses of AbstractFoo must set an instance attribute "
"self._bar in it's __init__ method")
class OkFoo(AbstractFoo):
def __init__(self):
self._bar = 3
class BadFoo(AbstractFoo):
pass
a = OkFoo()
b = BadFoo()
print a.bar
print b.bar # raises a NotImplementedError
This avoids the AttributeError: can't set attribute but if you just leave off the abstract property all together:
class AbstractFoo(object):
pass
class Foo(AbstractFoo):
pass
f = Foo()
f.bar
You get an AttributeError: 'Foo' object has no attribute 'bar' which is arguably almost as good as the NotImplementedError. So really my solution is just trading one error message from another .. and you have to use self._bar rather than self.bar in the init.
Following https://docs.python.org/2/library/abc.html you could do something like this in Python 2.7:
from abc import ABCMeta, abstractproperty
class Test(object):
__metaclass__ = ABCMeta
#abstractproperty
def test(self): yield None
def get_test(self):
return self.test
class TestChild(Test):
test = None
def __init__(self, var):
self.test = var
a = TestChild('test')
print(a.get_test())

dynamic class inheritance using super

I'm trying to dynamically create a class using type() and assign an __init__ constructor which calls super().__init__(...); however, when super() gets called I receive the following error:
TypeError: super(type, obj): obj must be an instance or subtype of type
Here is my code:
class Item():
def __init__(self, name, description, cost, **kwargs):
self.name = name
self.description = description
self.cost = cost
self.kwargs = kwargs
class ItemBase(Item):
def __init__(self, name, description, cost):
super().__init__(name, description, cost)
def __constructor__(self, n, d, c):
super().__init__(name=n, description=d, cost=c)
item = type('Item1', (ItemBase,), {'__init__':__constructor__})
item_instance = item('MyName', 'MyDescription', 'MyCost')
Why is super() inside the __constructor__ method not understanding the object parameter; and how do I fix it?
Solution 1: Using cls = type('ClassName', ...)
Note the solution of sadmicrowave creates an infinite loop if the dynamically-created class gets inherited as self.__class__ will correspond to the child class.
An alternative way which do not have this issue is to assigns __init__ after creating the class, such as the class can be linked explicitly through closure. Example:
# Base class
class A():
def __init__(self):
print('A')
# Dynamically created class
B = type('B', (A,), {})
def __init__(self):
print('B')
super(B, self).__init__()
B.__init__ = __init__
# Child class
class C(B):
def __init__(self):
print('C')
super().__init__()
C() # print C, B, A
Solution 2: Using MyClass.__name__ = 'ClassName'
An alternative way to dynamically create class is to define a class inside the function, then reassign the __name__ and __qualname__ attributes:
class A:
def __init__(self):
print(A.__name__)
def make_class(name, base):
class Child(base):
def __init__(self):
print(Child.__name__)
super().__init__()
Child.__name__ = name
Child.__qualname__ = name
return Child
B = make_class('B', A)
class C(B):
def __init__(self):
print(C.__name__)
super().__init__()
C() # Display C B A
Here is how I solved the issue. I reference the type() method to dynamically instantiate a class with variable references as such:
def __constructor__(self, n, d, c, h):
# initialize super of class type
super(self.__class__, self).__init__(name=n, description=d, cost=c, hp=h)
# create the object class dynamically, utilizing __constructor__ for __init__ method
item = type(item_name, (eval("{}.{}".format(name,row[1].value)),), {'__init__':__constructor__})
# add new object to the global _objects object to be used throughout the world
self._objects[ item_name ] = item(row[0].value, row[2].value, row[3].value, row[4].value)
There may be a better way to accomplish this, but I needed a fix and this is what I came up with... use it if you can.

Resources