I have a class named TEST in a configuration file config.py as follows:
# file config.py
class TEST():
alpha = 0.05
and a file run.py from which I try to access attributes of the class TEST through input argument as follows:
# file run.py
import sys
import config
if __name__ == "__main__":
conf = sys.argv[1] # comes from input argument conf=TEST
obj1 = config.TEST() # this works
print(obj1.alpha) # prints alpha variable 0.05
obj2 = config.conf() # AttributeError: module 'config' has no attribute 'conf'
print(obj2.alpha)
By running it as python3 run.py TEST, it gives attribute error in obj2, as it cannot convert the conf variable to TEST and access the class TEST. Any idea how to solve this?
Problem fixed by using the getattr from this thread:
Calling a function of a module by using its name (a string)
The code modification is as follows:
# file run.py
import sys
import config
if __name__ == "__main__":
conf = sys.argv[1] # comes from input argument conf=TEST
obj1 = config.TEST() # this works
print(obj1.alpha) # prints alpha variable 0.05
obj2 = getattr(config, conf)() #
print(obj2.alpha) # prints alpha variable 0.05
which by running it python3 run.py prints the desired.
Related
I have two python files file1.py and file2.py while accessing file2 variables in file1 I'm getting the error
file1.py contains:
from file2 import A
B = 10
print(A+B)
file2.py contains:
def func():
------ # ignore this function
------ # ignore this function
if '__name__'=='__main__':
A = 20
Then getting error :
AttributeError: module 'file2' has no attribute 'A'
How to access variables in other python file which are under
if '__name__'=='__main__':
I do not believe you can import variables under
if '__name__'=='__main__':
If the python interpreter is running that module (the source file) as the main program, it sets the special name variable to have a value “main”. If this file is being imported from another module, name will be set to the module’s name.
if __name__ == “main”: is used to execute some code only if the file was run directly, and not imported.
Instead, check this out:
if __name__ == "__main__":
A=20
else:
A=30
Now when you import file2 and print A, A will be 30.
g++ has this nice variable PRETTY_FUNCTION that contains the name of the function called. For functions it produces the prototype (e.g., "void foo(long)"). For member functions in classes, the name includes the class name (e.g., "void Foo::foo(long)").
Is there an equivalent in Python. I have been playing around with "sys._getframe()" which gets close, but there doesn't seem to be a similarly simple mechanism that includes the class name when it is a member function.
I like to use this in error handlers.
Surely someone has a magical one-liner for this...
TIA,
-Joe
Example Python Code
This is about the closest I can get right now. Search for "<<< here".
Typing Foo.CLNAME is kind of silly. It would be nice if this could somehow be collapsed to an external function and stay outside of the class. There must be a way to do this in Python in a way similar to the way LINE and NAME work.
The implementations of those two are based on code I got from here:
How to determine file, function and line number?
Nice stuff!
-Joe
=====
#!/usr/bin/env python
# import system modules
#
import os
import sys
# define a hack to get the function name
#
class __MYNAME__(object):
def __repr__(self):
try:
raise Exception
except:
return str(sys.exc_info()[2].tb_frame.f_back.f_code.co_name)
def __init__(self):
pass
__NAME__ = __MYNAME__()
# define a hack to get the line number in a program
#
class __MYLINE__(object):
def __repr__(self):
try:
raise Exception
except:
return str(sys.exc_info()[2].tb_frame.f_back.f_lineno)
__LINE__ = __MYLINE__()
# use these in a function call
#
def joe():
print("[FUNCTION] name: %s, line: %s" %
(__NAME__, __LINE__)) # <<< here
# use these in a class member function
#
class Foo():
def __init__(self):
Foo.__CLNAME__ = self.__class__.__name__
def joe(self):
print("[CLASS] name: %s::%s, line: %s" %
(Foo.__CLNAME__, __NAME__, __LINE__)) # <<< here
#--------------------------------
# test all this in a main program
#--------------------------------
def main(argv):
# main program
#
print("[MAIN PROGRAM] name: %s, line: %s" %
(__NAME__, __LINE__)) # <<< here
# function call
#
joe()
# class member function
#
foo = Foo()
foo.joe()
# exit gracefully
#
sys.exit(os.EX_OK)
#
# end of main
# begin gracefully
#
if __name__ == "__main__":
main(sys.argv[0:])
#
# end of file
I've written some python code that needs to read a config file at /etc/myapp/config.conf . I want to write a unit test for what happens if that file isn't there, or contains bad values, the usual stuff. Lets say it looks like this...
""" myapp.py
"""
def readconf()
""" Returns string of values read from file
"""
s = ''
with open('/etc/myapp/config.conf', 'r') as f:
s = f.read()
return s
And then I have other code that parses s for its values.
Can I, through some magic Python functionality, make any calls that readconf makes to open redirect to custom locations that I set as part of my test environment?
Example would be:
main.py
def _open_file(path):
with open(path, 'r') as f:
return f.read()
def foo():
return _open_file("/sys/conf")
test.py
from unittest.mock import patch
from main import foo
def test_when_file_not_found():
with patch('main._open_file') as mopen_file:
# Setup mock to raise the error u want
mopen_file.side_effect = FileNotFoundError()
# Run actual function
result = foo()
# Assert if result is expected
assert result == "Sorry, missing file"
Instead of hard-coding the config file, you can externalize it or parameterize it. There are 2 ways to do it:
Environment variables: Use a $CONFIG environment variable that contains the location of the config file. You can run the test with an environment variable that can be set using os.environ['CONFIG'].
CLI params: Initialize the module with commandline params. For tests, you can set sys.argv and let the config property be set by that.
In order to mock just calls to open in your function, while not replacing the call with a helper function, as in Nf4r's answer, you can use a custom patch context manager:
from contextlib import contextmanager
from types import CodeType
#contextmanager
def patch_call(func, call, replacement):
fn_code = func.__code__
try:
func.__code__ = CodeType(
fn_code.co_argcount,
fn_code.co_kwonlyargcount,
fn_code.co_nlocals,
fn_code.co_stacksize,
fn_code.co_flags,
fn_code.co_code,
fn_code.co_consts,
tuple(
replacement if call == name else name
for name in fn_code.co_names
),
fn_code.co_varnames,
fn_code.co_filename,
fn_code.co_name,
fn_code.co_firstlineno,
fn_code.co_lnotab,
fn_code.co_freevars,
fn_code.co_cellvars,
)
yield
finally:
func.__code__ = fn_code
Now you can patch your function:
def patched_open(*args):
raise FileNotFoundError
with patch_call(readconf, "open", "patched_open"):
...
You can use mock to patch a module's instance of the 'open' built-in to redirect to a custom function.
""" myapp.py
"""
def readconf():
s = ''
with open('./config.conf', 'r') as f:
s = f.read()
return s
""" test_myapp.py
"""
import unittest
from unittest import mock
import myapp
def my_open(path, mode):
return open('asdf', mode)
class TestSystem(unittest.TestCase):
#mock.patch('myapp.open', my_open)
def test_config_not_found(self):
try:
result = myapp.readconf()
assert(False)
except FileNotFoundError as e:
assert(True)
if __name__ == '__main__':
unittest.main()
You could also do it with a lambda like this, if you wanted to avoid declaring another function.
#mock.patch('myapp.open', lambda path, mode: open('asdf', mode))
def test_config_not_found(self):
...
I have a module that uses enums defined in the same package. I want to run it locally for self testing and also access it from other packages, but I can't get the import for the enum to handle both situations.
I have tried a blank _ _ init _ _.py (without the spaces) in both the sub and the proj directories, with no apparent change.
I have tried variations of python -m without success.
Here is the minimal code that shows my problems:
/my/proj
|----sub
| |---e1.py
| |---one.py
| |---two.py
|-p1.py
|-p2.py
----
$ cat /my/proj/sub/e1.py
from enum import Enum
class UsefulEnums(Enum):
ZERO = 0
----
$ cat /my/proj/sub/one.py
from e1 import UsefulEnums as E
def profound():
print('The value of {} is {}'.format(E.ZERO.name, E.ZERO.value))
if __name__ == '__main__':
profound()
/my/proj/sub$ python one.py
The value of ZERO is 0
----
$ cat /my/proj/sub/two.py
# note the dot before the module name. No other change from one
from .e1 import UsefulEnums as E
def profound():
print('The value of {} is {}'.format(E.ZERO.name, E.ZERO.value))
if __name__ == '__main__':
profound()
/proj/sub$ python two.py
Traceback (most recent call last):
File "two.py", line 1, in <module>
from .e1 import UsefulEnums as E
ModuleNotFoundError: No module named '__main__.e1'; '__main__' is not a package
----
$ cd /my/proj
/my/proj$ cat p1.py
import sub.one as a
if __name__ == '__main__':
a.profound()
/my/proj$ python p1.py
Traceback (most recent call last):
File "p1.py", line 1, in <module>
import sub.be1 as a
File "/home/esmipau/delete_me/proj/sub/one.py", line 1, in <module>
from e1 import UsefulEnums as E
ModuleNotFoundError: No module named 'e1'
----
/my/proj$ cat p2.py
import sub.two as a
if __name__ == '__main__':
a.profound()
/my/proj$ python p2.py
The value of ZERO is 0
When run from the 'sub' directory, one.py works as expected, two.py fails with a 'ModuleNotFoundError' error as detailed above.
When imported in parent code and run from the parent directory, two.py now works, and one.py fails with a different 'ModuleNotFoundError' error.
I would like a three.py in the 'sub' directory that uses the enums defined in e1.py, and which can be run locally for self testing etc, and can be included from external module not in the same directory.
--- edit for close as duplicate suggestions ---
This is not the same question as others that have been suggested such as Correct import and package structure now that __init__.py is optional as I require a way for one module to import another in the same directory regardless of whether the module is being executed locally or imported from another module.
I had simmilar issue and this solved it:
try creating __init__ file like this:
import my.proj.sub.e1
and then in file you want to use add:
from my.proj.sub import UsefulEnums as E
Given the structure :
/proj
|---/pkg
| |---> usefulEnums.py
| |---> usefulCode.py
|--> myProj.py
I want to be able to run usefulCode.py in self test mode and I want to import and use it in myProj.
$ cat proj/pkg/usefulEnums.py
from enum import Enum
class UsefulEnums(Enum):
ZERO = 0
$ cat proj/pkg/usefulCode.py
if __package__ is None:
# works with python <path/>tcp.py
from usefulEnums import UsefulEnums as E
else:
# works with python -m pkg.tcp
# or when imported by another module
from pkg.usefulEnums import UsefulEnums as E
# This works as well and appears more more portable and pythonic
# but the logic that makes it work is obtuse and fragile
# and it doesn't work if __package__ is None (even though it should!)
# from .usefulEnums import UsefulEnums as E
def profound():
print('The value of {} is {}'.format(E.ZERO.name, E.ZERO.value))
if __name__ == '__main__':
profound()
The main project code is as expected:
cat proj/myProj.py
from pkg.usefulEnums import UsefulEnums as E
import pkg.usefulCode as u
if __name__ == '__main__':
u.profound()
The above is the work around I am accepting as the answer to my stated question, but the reasoning eludes me.
I do not understand why the behavior of the 'dot' import does not work as expected when __package__ is not defined. Surely it always means import relative to the current package, and if no package is explicitly defined, then why doesn't it mean import from the same directory as the current module, which could be argued to be the currently default package?
AIM - I am trying to pass a config variable 'db_str' to my pytest script (test_script.py)
The db_str variable is defined in development.ini
I have tried using command
pytest -c development.ini regression_tests/test_script.py
But it didn't work
Error
> conn_string = config['db_string']
KeyError: 'db_string'
I tried using conftest.py, but didn't work
#contest.py code
import pytest
def pytest_addoption(parser):
parser.addoption("--set-db_st",
action="store",help="host='localhost' dbname='xyz' user='portaladmin'")
#pytest.fixture
def db_str(request):
return request.config.getoption("--set-db_str")
Pytest code
from S4M_pyramid.modelimport MyModel
from S4M_pyramid.lib.deprecated_pylons_globals import config
import subprocess
config['db_str'] = db_str
def test_get_dataset_mapping_id():
result = MyModel.get_dataset_mapping_id()
assert len(result) >1
How can I pass variable 'db_str' from development.ini or any other ini file to pytest script
The logic is the fillowing:
Define CLI argument that will be used to pass information about environment/config file
Get CLI argument value in pytest fixture
Parse config file
Use parsed config in get_database_string fixture to get database connection string
Use get_database_string fixture in your tests to get connection string
conftest.py
import os
from configparser import ConfigParser
# in root of the project there is file project_paths.py
# with the following code ROOT_DIR = os.path.dirname(os.path.abspath(__file__))
import project_paths
def pytest_addoption(parser):
"""Pytest hook that defines list of CLI arguments with descriptions and default values
:param parser: pytest specific argument
:return: void
"""
parser.addoption('--env', action='store', default='development',
help='setup environment: development')
#pytest.fixture(scope="function")
def get_database_string(get_config):
"""Fixture that returns db_string
:param get_config: fixture that returns ConfigParser object that access
to config file
:type: ConfigParser
:return: Returns database connection string
:rtype: str
"""
return get_config['<section name>']['db_string']
#pytest.fixture(scope="function")
def get_config(request):
"""Functions that reads and return ConfigParser object that access
to config file
:rtype: ConfigParser
"""
environment = request.config.getoption("--env")
config_parser = ConfigParser()
file_path = os.path.join(project_paths.ROOT_DIR, '{}.ini'.format(environment))
config_parser.read(file_path)
return config_parser
test_file.py
import pytest
def test_function(get_database_string)
print(get_database_string)
>>> <data base string from development.ini>
As described on pytest_addoption:
add options:
To add command line options, call parser.addoption(...).
To add ini-file values call parser.addini(...).
get options:
Options can later be accessed through the config object, respectively:
config.getoption(name) to retrieve the value of a command line option.
config.getini(name) to retrieve a value read from an ini-style file.
conftest.py:
def pytest_addoption(parser):
parser.addini('foo', '')
test.py:
def test_func(request):
request.config.getini('foo')