I wan't to do parallel with the files that is in the class DictionaryTagger.
I pass 5 files in the class,
class DictionaryTagger(object):
print_look = threading.Lock()
def __init__(self, dictionary_paths):
print('Hilo:', threading.current_thread().getName())
files = [open(path, 'r') for path in dictionary_paths]
dictionaries = [yaml.load(dict_file) for dict_file in files]
map(lambda x: x.close(), files)
self.dictionary = {}
self.max_key_size = 0
for curr_dict in dictionaries:
#print(curr_dict)
for key in curr_dict:
#print(type(curr_dict))
#self.dictionary =
if key in self.dictionary:
self.dictionary[key].extend(curr_dict[key])
else:
self.dictionary[key] = curr_dict[key]
self.max_key_size = max(self.max_key_size, len(key))
dictt = DictionaryTagger(['dicts/positive22.yml', 'dicts/negative22.yml', 'dicts/increasers.yml', 'dicts/decreasers.yml', 'dicts/inverter.yml'])
When I pass this, I have an error because ´dictionary_paths´ is not defined.
for i in range(3):
t = threading.Thread(target=DictionaryTagger)
t.demond = True
t.start()
init() missing 1 required positional argument: ´'dictionary_paths'´
You're receiving this error because the init function you defined as an argument ('dictionary_paths') without a default value. You can solve this in two different ways:
Add a default value to the init function
class DictionaryTagger(object):
...
def __init__(self, dictionary_paths=[]):
...
Provide an argument when starting your threads:
default_dictionary_paths = []
for i in range(3):
t = threading.Thread(target=DictionaryTagger, args=(default_dictionary_paths))
t.demond = True
t.start()
Related
I am absolutely new to Wrapper Classes in Python. I was trying to implement it in a program of linked list where multiple linked list are in use. My code is:
def nodewrap(cls):
class Nodewrap:
def __init__(self):
self.head = None
self.tail = None
return Nodewrap
#nodewrap
class Node:
def __init__(self,data):
self.data = data
self.next = None
class Intersection:
def addnode(self,d):
newnode = Node(d)
if head == None:
head = tail = newnode
else:
tail.next = newnode
tail = newnode
obj1 = Intersection()
obj2 = Intersection()
obj3 = Intersection()
s1 = int(input("Enter size of 1st list : \n"))
for i in range(s1):
obj1.addnode(int(input("Enter the data : \n")))
s2 = int(input("Enter size of 1st list : \n"))
for i in range(s2):
obj2.addnode(int(input("Enter the data : \n")))
temp1 = obj1.head
for i in range(s1):
temp2 = obj2.head
for j in range(s2):
if temp1.data == temp2.data:
obj3.addnode(temp1.data)
break
temp2 = temp2.next
temp1 = temp1.next
print("Intersection is :")
temp = obj3.head
while temp!=None:
print(temp.data,end=" ")
temp = temp.next
I thought of using a wrapper class to wrap the class Node instead of using objects of the class Intersection only with data fields as head, tail. But it is giving me some sort of error with regards to init().
Please help.
I was trying to learn it from here:
https://www.geeksforgeeks.org/wrapper-class-in-python/
I think I understand what you want to do, but I think that you don't want to use a decorator, but you want to inherit from NodeWrap class
class Nodewrap:
head = None
tail = None
class Node(NodeWrap):
def __init__(self,data):
self.data = data
self.next = None
But I don't see any reason why to inherit this way. This should be enough, for a linked list. I have added is_first and is_last property
from __future__ import annotations
class Node:
prev_node = None
next_node = None
def __init__(self, data):
self.data = data
def add_node(self, node: Node) -> None:
if self.prev_node is not None:
raise ValueError('Previous node already defined.')
self.next_node = node
node.prev_node = self
#property
def is_first(self) -> bool:
return self.prev_node is None
#property
def is_last(self) -> bool:
return self.next_node is None
You can implement next, iter and create an Iterator class.
I don't recommend using next as the variable name.
from __future__ import annotations reference here. It's just for self reference annotation.
I would like some help on dumping a custom class object to a YAML file. A representation of my class looks like below:
from classes.enum_classes import Priority, Deadline
class Test(yaml.YAMLObject):
yaml_tag = u'!RandomTestClass'
def __init__(self):
self._name = ""
self._request_id = ""
self._order = None
self._priority = Priority.medium
self._deadline = Deadline.label_c
The last two parameters are objects of different classes, both of which are Enum derived classes. I am trying to dump the contents of an object of Test class to a YAML output file. My __repr__ method for the Test class looks so:
def __repr__(self):
return "(name=%r, request=%r, order=%r, priority=%r, deadline=%r)" % \
((str(self.name) + str(self.order)), self.request_id, self.order,
self.priority.name, self._deadline.name)
Working off the constructors and representers section of the PyYAML documentation and the example provided there (especially considering that the authors use lists for some of the class variables), I would expect my YAML file to display the actual display tags in the __repr__ method, rather than the variable names themselves. This is what I see currently:
--- !ContainerClass
_requests: !!python/object/apply:collections.defaultdict
args:
- !!python/name:builtins.list ''
dictitems:
'108':
- !RandomTestClass
_deadline: &id002 !Deadline '3'
_name: '108.1'
_order: '1'
_priority: &id006 !Priority '1'
_request_id: '108'
- !RandomTestClass
_deadline: &id002 !Deadline '3'
_name: '108.2'
_order: '2'
_priority: &id006 !Priority '1'
_request_id: '108'
_name: TestContainer
I want it to look like so:
---
Requests:
- name: '108.1'
- requestID: '108'
- priority: '1'
- order: '1'
- deadline: '3'
- name: '108.2' <for the second entry in the list and so on>
Name: ContainerClass
No amount of fiddling around with the __repr__ method or anything else has resulted in the output I would like. So there are two issues I would love to get some help with.
How do I get a sane, readable representation? I am guessing I will have to write a custom representer, so if someone could guide me with some pointers, since I was not able to find much information on that any way.
Getting rid of those pointers, or whatever we would want to call them next to priority and deadline. Priority and Deadline are the two classes referenced in my __init___ method above, that are Enum subclasses. Since they are already subclasses, any attempts to subclass them to yaml.YAMLObject result in an error with mixins. To get around that, some posts suggested I do so:
class Priority(Enum):
low = 0
medium = 1
high = 2
class Deadline(Enum):
label_a = 0
label_b = 1
label_c = 2
label_d = 3
def priority_enum_representer(dumper, data):
return dumper.represent_scalar('!Priority', str(data.value))
def deadline_enum_representer(dumper, data):
return dumper.represent_scalar('!Deadline', str(data.value))
yaml.add_representer(Deadline, deadline_enum_representer)
yaml.add_representer(Priority, priority_enum_representer)
Any information/pointers on solving these two issues will be much much appreciated. Apologies for the long post, but I have learnt that more information generally leads to much more precise help.
UPDATE:
My YAML file is written based on a list of these RandomTestClass objects that are stored in a defaultdict(list) in a ContainerClass.
class ContainerClass(yaml.YAMLObject):
yaml_tag = u'ContainerClass'
def __init__(self):
self._name = ""
self._requests = defaultdict(list)
def __repr__(self):
return "(Name=%r, Requests=%r)" % \
(self._name, str(self._requests))
#property
def requests(self):
return self._requests
#requests.setter
def requests(self, new_req, value=None):
if type(new_req) is dict:
self._requests = new_req
else:
try:
self._requests[new_req].append(value)
except AttributeError:
# This means the key exists, but the value is a non-list
# entity. Change existing value to list, append new value
# and reassign to the same key
list_with_values \
= [self._requests[new_req], value]
self._requests[new_req] = list_with_values
The ContainerClass holds instances of Test objects. In another class, which is the entry point for my code containing __main__, I create multiple instances of Test objects, that are then stored in an ```ContainerClass`` object and dumped out to the YAML file.
# requisite imports here, based on
# how the files are stored
from classes.Test import Test
from classes.ContainerClass import ContainerClass
class RunTestClass:
if __name__ == '__main__':
yaml_container = ContainerClass()
test_object_a = Test()
test_object_a._name = '108.1'
test_object_a._order = 1
test_object_a._request_id = '108'
yaml_container._name = "TestContainer"
yaml_container._requests[test_object_a._request_id] = test_object_a
test_object_b = Test()
test_object_b._name = '108.2'
test_object_b._order = 2
test_object_b._request_id = '108'
yaml_container._name = "TestContainer"
yaml_container._requests[test_object_b._request_id] = test_object_b
with open(output_file, mode='w+') as outfile:
for test_class_object in yaml_container._requests:
yaml.dump(test_class_object, outfile, default_flow_style=False,
explicit_start=True, explicit_end=True)
UPDATE:
Adding a single, consolidated file to the question, executable to replicate the issue.
import yaml
from enum import Enum
from collections import defaultdict
class Priority(Enum):
low = 0
medium = 1
high = 2
class Deadline(Enum):
label_a = 0
label_b = 1
label_c = 2
label_d = 3
def priority_enum_representer(dumper, data):
return dumper.represent_scalar('!Priority', str(data.value))
def deadline_enum_representer(dumper, data):
return dumper.represent_scalar('!Deadline', str(data.value))
yaml.add_representer(Deadline, deadline_enum_representer)
yaml.add_representer(Priority, priority_enum_representer)
class Test(yaml.YAMLObject):
yaml_tag = u'!RandomTestClass'
def __init__(self):
self._name = ""
self._request_id = ""
self._order = None
self._priority = Priority.medium
self._deadline = Deadline.label_c
#property
def name(self):
return self._name
#name.setter
def name(self, name):
self._name = name
#property
def request_id(self):
return self._request_id
#request_id.setter
def request_id(self, r_id):
self._request_id = r_id
#property
def order(self):
return self._order
#order.setter
def order(self, order):
self._order = order
#property
def priority(self):
return self._priority
#priority.setter
def priority(self, priority):
self._priority = priority
#property
def deadline(self):
return self._deadline
#deadline.setter
def deadline(self, deadline):
self._deadline = deadline
def __str__(self):
return self.name + ", " + self._request_id + ", " + str(self.order) + ", " \
+ str(self.priority) + ", " + str(self.deadline)
class ContainerClass(yaml.YAMLObject):
yaml_tag = u'ContainerClass'
def __init__(self):
self._name = ""
self._requests = defaultdict(list)
def __repr__(self):
return "(Name=%r, Requests=%r)" % \
(self._name, str(self._requests))
#property
def name(self):
return self._name
#name.setter
def name(self, name):
self._name = name
#property
def requests(self):
return self._requests
def set_requests(self, new_req, value=None):
if type(new_req) is dict:
self._requests = new_req
else:
try:
self._requests[new_req].append(value)
except AttributeError:
# This means the key exists, but the value is a non-list
# entity. Change existing value to list, append new value
# and reassign to the same key
print("Encountered a single value, converting to a list and appending new value")
list_with_values \
= [self._requests[new_req], value]
self._requests[new_req] = list_with_values
yaml_container = ContainerClass()
yaml_container.name = "TestContainer"
test_object_a = Test()
test_object_a._name = '108.1'
test_object_a._order = 1
test_object_a._request_id = '108'
yaml_container.set_requests(test_object_a.request_id, test_object_a)
test_object_b = Test()
test_object_b._name = '108.2'
test_object_b._order = 2
test_object_b._request_id = '108'
yaml_container.set_requests(test_object_b.request_id, test_object_b)
with open('test.yaml', mode='w+') as outfile:
yaml.dump(yaml_container, outfile, default_flow_style=False,
explicit_start=True, explicit_end=True)
There are different ways to solve this. Generally, you want to define a classmethod to_yaml in ContainerClass that will be called by the dumper, and implement the transition from native structure to YAML node graph there.
A minimal solution would be to just construct the structure you want to have as normal dict with a list in it, and tell the dumper to use that instead of the real structure:
#classmethod
def to_yaml(cls, dumper, data):
rSeq = []
for value in data._requests.values():
for t in value:
rSeq.extend([
{"name": t._name},
{"requestID": t._request_id},
{"priority": t._priority},
{"order": t._order},
{"deadline": t._deadline}
])
return dumper.represent_mapping("tag:yaml.org,2002:map",
{"Requests": rSeq, "Name": data._name})
This will give you
---
Name: TestContainer
Requests:
- name: '108.1'
- requestID: '108'
- priority: &id001 !Priority '1'
- order: 1
- deadline: &id002 !Deadline '2'
- name: '108.2'
- requestID: '108'
- priority: *id001
- order: 2
- deadline: *id002
...
YAML generates anchors & aliases for priority and requestID because the values refer the same objects. You can avoid those by doing
yaml.Dumper.ignore_aliases = lambda *args : True
before you dump the YAML.
You can be more sophisticated and iterate the properties instead of hard coding the names, but that is beyond the scope of this answer. If you want to load this YAML again into the same structure, you will need to add another classmethod from_yaml implementing the reverse transformation.
I am trying to write a simple code in Python. I'm still weak at the fundamentals of Python and I'm stuck on this issue. I have an inner Class Node(). It has an attribute routeNodes in it which is a list. I am using it to store other Instances of the same Class Object(Node()). So I first initialize the Class firstNode and give it some dummy values.
Then I run a for loop for 10 iterations. In each of the iterations I create another instance of the same Node() class. I then append the firstNode object (which is again an instance of Node()) to each of these secondNode classes.
It adds the firstNode to the routeNode list variable of the secondeNode.
But it also adds firstnode object to the routenode list variable of its own class object!
The same firstNode Object gets appended to both the class objects. Why does this happen? Can I not add class objects like this? and how do I stop it from happening?
Attaching my code. You can run it in a test class and debug the "secondNode.addRouteNodes(firstNode)" line and you'll understand my problem.
class Node:
kms = 0
cumulativeKms = 0
routeNodes = []
nodeName = ""
def __init__(self, nodeName):
self.nodeName = nodeName
def setNodeName(self, nodeName):
self.nodeName = nodeName
def getNodeName(self):
return self.nodeName
def setKms(self, kms):
self.kms = kms
def getKms(self):
return self.kms
def setCumulativeKms(self, kms):
self.cumulativeKms = self.cumulativeKms + kms
def getCumulativeKms(self):
return self.cumulativeKms
def addRouteNodes(self, nodes):
self.routeNodes.append(nodes)
def getRouteNodes(self):
return self.routeNodes
firstNode = Node("first")
firstNode.setKms(0)
firstNode.setCumulativeKms(0)
for i in range(10):
secondNode = Node('new'+str(i))
secondNode.setKms(10)
secondNode.setCumulativeKms(10)
routes = firstNode.getRouteNodes()
for route in routes:
secondNode.addRouteNodes(route)
secondNode.addRouteNodes(firstNode)
print("Done")
class Node:
kms = 0
cumulativeKms = 0
routeNodes = []
nodeName = ""
These variables don't belong to each new class object created but rather are shared between the objects and hence the first class gets appended to its own routeNodes variable over here : secondNode.addRouteNodes(route).They are called class variables.
A solution would be to initialize the variables inside the constructor like below :
self.kms = 0
self.cumulativeKms = 0
self.routeNodes = []
self.nodeName = ""
These are called instance variables.You need this.
So the updated code would be :
class Node:
def __init__(self):
self.kms = 0
self.cumulativeKms = 0
self.routeNodes = []
self.nodeName = ""
def setNodeName(self, nodeName):
self.nodeName = nodeName
def getNodeName(self):
return self.nodeName
def setKms(self, kms):
self.kms = kms
def getKms(self):
return self.kms
def setCumulativeKms(self, kms):
self.cumulativeKms = self.cumulativeKms + kms
def getCumulativeKms(self):
return self.cumulativeKms
def addRouteNodes(self, nodes):
self.routeNodes.append(nodes)
def getRouteNodes(self):
return self.routeNodes
firstNode = Node()
firstNode.setNodeName("first")
firstNode.setKms(0)
firstNode.setCumulativeKms(0)
for i in range(10):
secondNode = Node()
secondNode.setNodeName('new'+str(i))
secondNode.setKms(10)
secondNode.setCumulativeKms(10)
routes = firstNode.getRouteNodes()
for route in routes:
secondNode.addRouteNodes(route)
secondNode.addRouteNodes(firstNode)
print("Done")
I have written a custom logging class for module logging that I called call. With this class I hope to place it in any function/method and it logs the function name with its arguments and all values the function was called with.
This works fine for class methods
Foo.bar(self, a=1, b=2, c=3, *args=(), **kwargs={'something': 4})
using this minimal example
import logging
import inspect
def call(logger):
fname = [] # Function name including module and class
fargs = [] # Arguments of function including positional and named arguments
parentframe = inspect.stack()[1][0]
module = inspect.getmodule(parentframe)
if module and module.__name__ != "__main__":
fname.append(module.__name__)
codename = parentframe.f_code.co_name
if "self" in parentframe.f_locals:
fname.append(parentframe.f_locals["self"].__class__.__name__)
fobj = getattr(parentframe.f_locals["self"].__class__, codename)
if codename != "<module>":
fname.append(codename)
argspec = inspect.formatargspec(*inspect.getfullargspec(fobj))
args = argspec[1:-1].split(",")
for arg in args:
argkey = arg.strip().replace("*", "").split("=")[0]
if arg == "self":
fargs.append("self")
else:
fargs.append(arg.split("=")[0] + "=" + str(parentframe.f_locals[argkey]))
del parentframe
msg = ".".join(fname) + "(" + ",".join(fargs) + ")"
if logger.isEnabledFor(30):
logger.log(30, msg)
class Foo:
def __init__(self, l):
self.logger = l
def bar(self, a, b, c=3, *args, **kwargs):
call(self.logger)
if __name__ == "__main__":
logging.addLevelName(30, "CALL")
logger = logging.getLogger('blub')
logger.level = 20
f = Foo(logger)
f.bar(1, 2, something=4)
print("done...")
My problem is when I use the same functionality on static methods or simple functions. It fails at the line where I get the function object (fobj = getattr(parentframe.f_locals["self"].__class__, codename)
) using self.
parentframe is the Frame object of the function in questions I presume. I have not yet found a way to get the function object from that object. Is there a way?
Use getattr(module, codename) to get the function-object of functions that are not contained in classes.
Here the full code:
import logging
import inspect
def call(logger):
fname = [] # Function name including module and class
fargs = [] # Arguments of function including positional and named arguments
parentframe = inspect.stack()[1][0]
module = inspect.getmodule(parentframe)
if module and module.__name__ != "__main__":
fname.append(module.__name__)
codename = parentframe.f_code.co_name
if "self" in parentframe.f_locals:
fname.append(parentframe.f_locals["self"].__class__.__name__)
fobj = getattr(parentframe.f_locals["self"].__class__, codename)
else:
fobj = getattr(module, codename)
if codename != "<module>":
fname.append(codename)
argspec = inspect.formatargspec(*inspect.getfullargspec(fobj))
args = argspec[1:-1].split(",")
for arg in args:
argkey = arg.strip().replace("*", "").split("=")[0]
if arg == "self":
fargs.append("self")
else:
fargs.append(arg.split("=")[0] + "=" + str(parentframe.f_locals[argkey]))
del parentframe
msg = ".".join(fname) + "(" + ",".join(fargs) + ")"
if logger.isEnabledFor(30):
logger.log(30, msg)
class Foo:
def __init__(self, l):
self.logger = l
def bar(self, a, b, c=3, *args, **kwargs):
call(self.logger)
def boo(a, b, c=3, *args, **kwargs):
call(logger)
if __name__ == "__main__":
logging.addLevelName(30, "CALL")
logger = logging.getLogger('blub')
logger.level = 20
f = Foo(logger)
f.bar(1, 2, something=4)
boo(1, 2, something=4)
print("done...")
Taking ideas from both, I wrote this function to find the function object from a frame. I'm sure there's some edge cases around inherited staticmethods, and obviously any code not using the cls and self conventions for param names. This also doesn't work for lambdas... but you shouldn't be logging anything out in a lamba anyway :-P
def _get_func_obj(f):
"""
Get function object from a frame. If it can't find it, return None
"""
codename = f.f_code.co_name
fobj = None
try:
if "self" in f.f_locals: # regular method
fobj = getattr(f.f_locals["self"].__class__, codename)
elif "cls" in f.f_locals: # class method
fobj = getattr(f.f_locals["cls"], codename)
else:
module = inspect.getmodule(f) # only fetch module if we need it
if hasattr(module, codename): # regular module level function
fobj = getattr(module, codename)
else: # static method
classes = [
getattr(module, name)
for name in dir(module)
if inspect.isclass(getattr(module, name))
]
for cls in classes:
if (
hasattr(cls, codename)
and getattr(cls, codename).__code__ == f.f_code
):
fobj = getattr(cls, codename)
break
if fobj is None:
"""it's likely some nested function/method or a lambda, who logs in a lambda?"""
return fobj
except Exception:
"""never break logging"""
I get the following error from my code: TypeError: get_manifest() takes 0 positional arguments but 1 was given
I have a class scan, and get_manifest is a function in scan:
import amfy
import requests
import json
from datetime import timedelta, datetime
class Scan:
def __init__(self, realm = 416, cluster = 9):
self.events = list()
self.url = "http://realm{}.c{}0.castle.rykaiju.com/api/locales/en/get_serialized_new".format(realm, cluster)
def get_manifest():
self.request = requests.get(self.url)
if self.request.status_code == 200:
self.manifest = self.request.json() if "json" in self.request.headers["content-type"] else amfy.loads(self.request.content)
else:
self.manifest = {"failed": req.reason}
def get_time(day):
self.today = datetime.now()
self.plusdays = timedelta(days = day)
self.new_date = sel.ftoday + self.plusdays
self.year, self.month, self.day = self.new_date.year, self.new_date.month, self.new_date.day
if len(str(self.day)) == 1:
day = "0{}".format(self.day)
if len(str(self.month)) == 1:
month = "0{}".format(self.month)
self.date = str(self.year) + str(self.month) + str(self.day)
return self.date
def search_events():
for day in range(0, 11):
date = self.get_time(day)
for section in doaManifest:
for key, value in doaManifest[section].items():
if date in key:
event_title = key
try:
event_desc = value['text']
except KeyError:
event_desc = """" /!\ No Event Description /!\""""
my_dict = {'title' : event_title, 'contents' : event_desc}
self.events.append(my_dict)
Then, in another class, which is my app GUI (written with tkinter), I have a button that calls on this class. The button's command is this:
def scan(self):
if self.scan['text'] == 'Scan':
self.scan['text'] = 'Re-Scan'
self.scanned = True
self.scan = Scan()
self.scan.get_manifest()
self.scan.search_events()
I don't feed get_manifest any arguments, so why is it saying I am?
Seems like you forgot adding self to get_manifest(). Redefine get_manifest() like so:
def get_manifest(self):
code...
Instance methods should always accept self as the first argument as it's automatically inserted. You may use #staticmethod if you don't wish to receive it but as you're using self I guess that it's not something you want.
First you need to add the self parameter to all class methods unless you use #staticmethod. So your class should look like this:
class Scan:
def __init__(self, realm = 416, cluster = 9):
# code...
def get_manifest(self):
# code...
def get_time(self, day):
# code...
def search_events(self):
# code...
Furthermore, to use this class within your scan() method you need to initialize and save the class instance to a variable, then call on the get_manifest() method from the saved instance variable.
def scan(self):
# I assume this scan() method is actually a part of a class since 'self' is used
if self.scan['text'] == 'Scan':
self.scan['text'] = 'Re-Scan'
self.scanned = True
#self.scan = Scan() # You are overwriting the definition of 'scan' here...
# So instead do the below
# Initialize and save your Scan() instance to a variable
scan_instance = Scan()
# Now you may use the object's methods
scan_instance.get_manifest()
scan_instance.search_events()
# Here you can now access the object's attributes that were created in get_manifest()
print(scan_instance.manifest)