Dumping custom class objects to a YAML file - python-3.x

Question

I would like some help on dumping a custom class object to a YAML file. A representation of my class looks like below:
from classes.enum_classes import Priority, Deadline
class Test(yaml.YAMLObject):
yaml_tag = u'!RandomTestClass'
def __init__(self):
self._name = ""
self._request_id = ""
self._order = None
self._priority = Priority.medium
self._deadline = Deadline.label_c
The last two parameters are objects of different classes, both of which are Enum derived classes. I am trying to dump the contents of an object of Test class to a YAML output file. My __repr__ method for the Test class looks so:
def __repr__(self):
return "(name=%r, request=%r, order=%r, priority=%r, deadline=%r)" % \
((str(self.name) + str(self.order)), self.request_id, self.order,
self.priority.name, self._deadline.name)
Working off the constructors and representers section of the PyYAML documentation and the example provided there (especially considering that the authors use lists for some of the class variables), I would expect my YAML file to display the actual display tags in the __repr__ method, rather than the variable names themselves. This is what I see currently:
--- !ContainerClass
_requests: !!python/object/apply:collections.defaultdict
args:
- !!python/name:builtins.list ''
dictitems:
'108':
- !RandomTestClass
_deadline: &id002 !Deadline '3'
_name: '108.1'
_order: '1'
_priority: &id006 !Priority '1'
_request_id: '108'
- !RandomTestClass
_deadline: &id002 !Deadline '3'
_name: '108.2'
_order: '2'
_priority: &id006 !Priority '1'
_request_id: '108'
_name: TestContainer
I want it to look like so:
---
Requests:
- name: '108.1'
- requestID: '108'
- priority: '1'
- order: '1'
- deadline: '3'
- name: '108.2' <for the second entry in the list and so on>
Name: ContainerClass
No amount of fiddling around with the __repr__ method or anything else has resulted in the output I would like. So there are two issues I would love to get some help with.
How do I get a sane, readable representation? I am guessing I will have to write a custom representer, so if someone could guide me with some pointers, since I was not able to find much information on that any way.
Getting rid of those pointers, or whatever we would want to call them next to priority and deadline. Priority and Deadline are the two classes referenced in my __init___ method above, that are Enum subclasses. Since they are already subclasses, any attempts to subclass them to yaml.YAMLObject result in an error with mixins. To get around that, some posts suggested I do so:
class Priority(Enum):
low = 0
medium = 1
high = 2
class Deadline(Enum):
label_a = 0
label_b = 1
label_c = 2
label_d = 3
def priority_enum_representer(dumper, data):
return dumper.represent_scalar('!Priority', str(data.value))
def deadline_enum_representer(dumper, data):
return dumper.represent_scalar('!Deadline', str(data.value))
yaml.add_representer(Deadline, deadline_enum_representer)
yaml.add_representer(Priority, priority_enum_representer)
Any information/pointers on solving these two issues will be much much appreciated. Apologies for the long post, but I have learnt that more information generally leads to much more precise help.
UPDATE:
My YAML file is written based on a list of these RandomTestClass objects that are stored in a defaultdict(list) in a ContainerClass.
class ContainerClass(yaml.YAMLObject):
yaml_tag = u'ContainerClass'
def __init__(self):
self._name = ""
self._requests = defaultdict(list)
def __repr__(self):
return "(Name=%r, Requests=%r)" % \
(self._name, str(self._requests))
#property
def requests(self):
return self._requests
#requests.setter
def requests(self, new_req, value=None):
if type(new_req) is dict:
self._requests = new_req
else:
try:
self._requests[new_req].append(value)
except AttributeError:
# This means the key exists, but the value is a non-list
# entity. Change existing value to list, append new value
# and reassign to the same key
list_with_values \
= [self._requests[new_req], value]
self._requests[new_req] = list_with_values
The ContainerClass holds instances of Test objects. In another class, which is the entry point for my code containing __main__, I create multiple instances of Test objects, that are then stored in an ```ContainerClass`` object and dumped out to the YAML file.
# requisite imports here, based on
# how the files are stored
from classes.Test import Test
from classes.ContainerClass import ContainerClass
class RunTestClass:
if __name__ == '__main__':
yaml_container = ContainerClass()
test_object_a = Test()
test_object_a._name = '108.1'
test_object_a._order = 1
test_object_a._request_id = '108'
yaml_container._name = "TestContainer"
yaml_container._requests[test_object_a._request_id] = test_object_a
test_object_b = Test()
test_object_b._name = '108.2'
test_object_b._order = 2
test_object_b._request_id = '108'
yaml_container._name = "TestContainer"
yaml_container._requests[test_object_b._request_id] = test_object_b
with open(output_file, mode='w+') as outfile:
for test_class_object in yaml_container._requests:
yaml.dump(test_class_object, outfile, default_flow_style=False,
explicit_start=True, explicit_end=True)
UPDATE:
Adding a single, consolidated file to the question, executable to replicate the issue.
import yaml
from enum import Enum
from collections import defaultdict
class Priority(Enum):
low = 0
medium = 1
high = 2
class Deadline(Enum):
label_a = 0
label_b = 1
label_c = 2
label_d = 3
def priority_enum_representer(dumper, data):
return dumper.represent_scalar('!Priority', str(data.value))
def deadline_enum_representer(dumper, data):
return dumper.represent_scalar('!Deadline', str(data.value))
yaml.add_representer(Deadline, deadline_enum_representer)
yaml.add_representer(Priority, priority_enum_representer)
class Test(yaml.YAMLObject):
yaml_tag = u'!RandomTestClass'
def __init__(self):
self._name = ""
self._request_id = ""
self._order = None
self._priority = Priority.medium
self._deadline = Deadline.label_c
#property
def name(self):
return self._name
#name.setter
def name(self, name):
self._name = name
#property
def request_id(self):
return self._request_id
#request_id.setter
def request_id(self, r_id):
self._request_id = r_id
#property
def order(self):
return self._order
#order.setter
def order(self, order):
self._order = order
#property
def priority(self):
return self._priority
#priority.setter
def priority(self, priority):
self._priority = priority
#property
def deadline(self):
return self._deadline
#deadline.setter
def deadline(self, deadline):
self._deadline = deadline
def __str__(self):
return self.name + ", " + self._request_id + ", " + str(self.order) + ", " \
+ str(self.priority) + ", " + str(self.deadline)
class ContainerClass(yaml.YAMLObject):
yaml_tag = u'ContainerClass'
def __init__(self):
self._name = ""
self._requests = defaultdict(list)
def __repr__(self):
return "(Name=%r, Requests=%r)" % \
(self._name, str(self._requests))
#property
def name(self):
return self._name
#name.setter
def name(self, name):
self._name = name
#property
def requests(self):
return self._requests
def set_requests(self, new_req, value=None):
if type(new_req) is dict:
self._requests = new_req
else:
try:
self._requests[new_req].append(value)
except AttributeError:
# This means the key exists, but the value is a non-list
# entity. Change existing value to list, append new value
# and reassign to the same key
print("Encountered a single value, converting to a list and appending new value")
list_with_values \
= [self._requests[new_req], value]
self._requests[new_req] = list_with_values
yaml_container = ContainerClass()
yaml_container.name = "TestContainer"
test_object_a = Test()
test_object_a._name = '108.1'
test_object_a._order = 1
test_object_a._request_id = '108'
yaml_container.set_requests(test_object_a.request_id, test_object_a)
test_object_b = Test()
test_object_b._name = '108.2'
test_object_b._order = 2
test_object_b._request_id = '108'
yaml_container.set_requests(test_object_b.request_id, test_object_b)
with open('test.yaml', mode='w+') as outfile:
yaml.dump(yaml_container, outfile, default_flow_style=False,
explicit_start=True, explicit_end=True)

Answer 1

There are different ways to solve this. Generally, you want to define a classmethod to_yaml in ContainerClass that will be called by the dumper, and implement the transition from native structure to YAML node graph there.
A minimal solution would be to just construct the structure you want to have as normal dict with a list in it, and tell the dumper to use that instead of the real structure:
#classmethod
def to_yaml(cls, dumper, data):
rSeq = []
for value in data._requests.values():
for t in value:
rSeq.extend([
{"name": t._name},
{"requestID": t._request_id},
{"priority": t._priority},
{"order": t._order},
{"deadline": t._deadline}
])
return dumper.represent_mapping("tag:yaml.org,2002:map",
{"Requests": rSeq, "Name": data._name})
This will give you
---
Name: TestContainer
Requests:
- name: '108.1'
- requestID: '108'
- priority: &id001 !Priority '1'
- order: 1
- deadline: &id002 !Deadline '2'
- name: '108.2'
- requestID: '108'
- priority: *id001
- order: 2
- deadline: *id002
...
YAML generates anchors & aliases for priority and requestID because the values refer the same objects. You can avoid those by doing
yaml.Dumper.ignore_aliases = lambda *args : True
before you dump the YAML.
You can be more sophisticated and iterate the properties instead of hard coding the names, but that is beyond the scope of this answer. If you want to load this YAML again into the same structure, you will need to add another classmethod from_yaml implementing the reverse transformation.