How to get a Hydra config without using #hydra.main() - python-3.x

Question

Let's say we have following setup (copied & shortened from the Hydra docs):
Configuration file: config.yaml
db:
driver: mysql
user: omry
pass: secret
Python file: my_app.py
import hydra
#hydra.main(config_path="config.yaml")
def my_app(cfg):
print(cfg.pretty())
if __name__ == "__main__":
my_app()
This works well when we can use a decorator on the function my_app. Now I would like (for small scripts and testing purposes, but that is not important) to get this cfg object outside of any function, just in a plain python script. From what I understand how decorators work, it should be possible to call
import hydra
cfg = hydra.main(config_path="config.yaml")(lambda x: x)()
print(cfg.pretty())
but then cfg is just None and not the desired configuration object. So it seems that the decorator does not pass on the returned values. Is there another way to get to that cfg ?

Answer 1

Use the Compose API:
from hydra import compose, initialize
from omegaconf import OmegaConf
initialize(config_path="conf", job_name="test_app")
cfg = compose(config_name="config", overrides=["db=mysql", "db.user=me"])
print(OmegaConf.to_yaml(cfg))
This will only compose the config and will not have side effects like changing the working directory or configuring the Python logging system.

Answer 2

None of the above solutions worked for me. They gave errors:
'builtin_function_or_method' object has no attribute 'code'
and
GlobalHydra is already initialized, call
Globalhydra.instance().clear() if you want to re-initialize
I dug further into hydra and realised I could just use OmegaConf to load the file directly. You don't get overrides but I'm not fussed about this.
import omegaconf
cfg = omegaconf.OmegaConf.load(path)

Answer 3

I found a rather ugly answer but it works - if anyone finds a more elegant solution please let us know!
We can use a closure or some mutable object. In this example we define a list outside and append the config object:
For hydra >= 1.0.0 you have to use config_name instead, see documentation.
import hydra
c = []
hydra.main(config_name="config.yaml")(lambda x:c.append(x))()
cfg = c[0]
print(cfg)
For older versions:
import hydra
c = []
hydra.main(config_path="config.yaml")(c.append)()
cfg = c[0]
print(cfg.pretty())

Answer 4

anther ugly answer, but author said this may be crush in next version
Blockquote
from omegaconf import DictConfig
from hydra.utils import instantiate
from hydra._internal.utils import _strict_mode_strategy, split_config_path, create_automatic_config_search_path
from hydra._internal.hydra import Hydra
from hydra.utils import get_class
class SomeThing:
...
def load_from_yaml(self, config_path, strict=True):
config_dir, config_file = split_config_path(config_path)
strict = _strict_mode_strategy(strict, config_file)
search_path = create_automatic_config_search_path(
config_file, None, config_dir
)
hydra = Hydra.create_main_hydra2(
task_name='sdfs', config_search_path=search_path, strict=strict
)
config = hydra.compose_config(config_file, [])
config.pop('hydra')
self.config = config
print(self.config.pretty())

Answer 5

This is my solution
from omegaconf import OmegaConf
class MakeObj(object):
""" dictionary to object.
Thanks to https://stackoverflow.com/questions/1305532/convert-nested-python-dict-to-object
Args:
object ([type]): [description]
"""
def __init__(self, d):
for a, b in d.items():
if isinstance(b, (list, tuple)):
setattr(self, a, [MakeObj(x) if isinstance(x, dict) else x for x in b])
else:
setattr(self, a, MakeObj(b) if isinstance(b, dict) else b)
def read_yaml(path):
x_dict = OmegaConf.load(path)
x_yamlstr = OmegaConf.to_yaml(x_dict)
x_obj = MakeObj(x_dict)
return x_yamlstr, x_dict, x_obj
x_yamlstr, x_dict, x_obj = read_yaml('config/train.yaml')
print(x_yamlstr)
print(x_dict)
print(x_obj)
print(dir(x_obj))