I have the following project structure in Pycharm -
project/
src/
main.py
config/
__init__.py
settings.py
package1/
__init__.py
client.py # Imports settings from config using -> from ..config import settings
tests/
__init__.py
test.py
So here my client.py imports settings -
from ..config import settings
main.py imports client -
from package1.client import clientFunction # ClientFunction is the method defined inside client.py
I use pytest to run the tests. So now when i run the pytest configuration, in my test cases i import main. I dont see any errors in my code but when the tests run it says -
from ..config import settings
E ValueError: attempted relative import beyond top-level package
How can i fix this? Seems like pytest is taking a different path to execute the tests and hence the relative imports are going beyond the top level.
I have a python package i'm working on with the below structure:
package/
__init__.py
script1.py
script2.py
script3.py
all 3 scripts have common imports for example:
import sys
import socket
import subprocess
i think it would be more efficient and also 'prettier' if those common imports could be done in the __init__.py module rather being imported time after time in each script. I've tried to move those imports to the init module , but then they're no longer recognized in their respective python scripts.
What would be the best approach to unify common imports for a python package?
I am new to Python. This really confused me!
My directory structure is like this:
Project
| - subpackage1
|- a.py
| - subpackage2
|- b.py
| - c.py
When I import a.py into b.py with from subpackage1 import a, I get a ModuleNotFoundError. It seems like I cannot import a file from the parent directory.
Some solutions suggest to add an empty file __init__.py in each directory, but that does not work. As a work around, I have put the following in each subfile (i.e., a.py and b.py) to access the parent directory:
import os
import sys
sys.path.append(os.path.abspath('..'))
I have tried to output sys.path in subfiles, it only includes the current file path and anaconda path, so I have to append .. to sys.path.
How can I solve this problem? Is there a more efficient way?
Suppose we have this files and directories tree:
$> tree
.
├── main.py
├── submodule1
│ ├── a.py
│ └── __init__.py
└── submodule2
├── b.py
└── __init__.py
2 directories, 5 files
So, here is an example of how to do the import from a.py inti b.py and vice-versa.
a.py
try:
# Works when we're at the top lovel and we call main.py
from submodule1 import b
except ImportError:
# If we're not in the top level
# And we're trying to call the file directly
import sys
# add the submodules to $PATH
# sys.path[0] is the current file's path
sys.path.append(sys.path[0] + '/..')
from submodule2 import b
def hello_from_a():
print('hello from a')
if __name__ == '__main__':
hello_from_a()
b.hello_from_b()
b.py
try:
from submodule1 import a
except ImportError:
import sys
sys.path.append(sys.path[0] + '/..')
from submodule1 import a
def hello_from_b():
print("hello from b")
if __name__ == '__main__':
hello_from_b()
a.hello_from_a()
And, main.py:
from submodule1 import a
from submodule2 import b
def main():
print('hello from main')
a.hello_from_a()
b.hello_from_b()
if __name__ == '__main__':
main()
Demo:
When we're in the top level and we're trying to call main.py
$> pwd
/home/user/modules
$> python3 main.py
hello from main
hello from a
hello from b
When w're in /modules/submodule1 level and we're trying to call a.py
$> pwd
/home/user/modules/submodule1
$> python3 a.py
hello from a
hello from b
When we're /modules/submodule2 level and we're trying to call b.py
$> pwd
/home/user/modules/submodule2
$> python3 b.py
hello from b
hello from a
The first issue you're getting is due to that from subpackage1 import a line in your b.py module.
b.py is located in your subpackage2 package, a sibling package of subpackage1. So trying to run from subpackage1 import a means the subpackage1 is within subpackage2 which is incorrect. Also as side note, in python3 you should never use implicit relative imports as it no longer supports it, so instead use explicit relative imports.
Adding an __init__.py in each folder turns them in python packages, and you can keep them empty. You want to replace the from subpackage1 import a with either an explicit relative import like from ..subpackage1 import a or an absolute import like from Project.subpackage1 import a. This will be the efficient and correct way to write your package, and you can test it by writing a script that imports Project and uses its subpackages and modules.
However, I am assuming you are running b.py as a main module to test the imports. This means you are running command lines that look like: python b.py. Running it like this gives you a module search path that does not have any of the parent paths like Project. This would lead to you keep getting ModuleNotFoundErrors even though there's nothing technically wrong with your package. This is why you need a sys.path.append(... work-around that manually appends your parent path to the module search path in order to run your b.py module as a main module. If this helps you to test your code then by all means use it, but it is fine to use the absolute and explicit relative imports because modules in a package are supposed to work that way.
Firstly, you need create file __init__.py in subpackage1 to declare it is a module
touch subpackage1/__init__.py
Secondly, you can try relative import in python3
# b.py
from ..subpackage1 import a
Or you can add your current directory to $PYTHONPATH
export PYTHONPATH=${PYTHONPATH}:${PWD}
One way to access the subpackage is using . operator going all the way up to top package or file directory. Thus if the structure is
top_directory
|- package1
|- subpackage1
|- a.py
|- package2
|- subpackage2
|- b.py
Then you use the following
#b.py
from top_directory.package1.subpackage1 import a
statements...
The from statement must go all the way up to the top directory covering both a.py and b.py.
This SO Q&A discusses options for adding your project root to the PYTHONPATH.
1. mac/linux:
add export PYTHONPATH="${PYTHONPATH}:/path/to/Project_top_directory" to your ~/.bashrc
2. path config file:
Alternatively, you can create a path configuration file
# find directory
SITEDIR=$(python -m site --user-site)
# create if it doesn't exist
mkdir -p "$SITEDIR"
# create new .pth file with our path
echo "$HOME/foo/bar" > "$SITEDIR/somelib.pth"
I would like to import a class from my submodule without having to use the from submodule.submodule import Class syntax. Instead I would just like to do from submodule import Class like a normal Python3 module.
I feel this should have been answered a million times, and while there are several similarly named questions on SO, none of them provide a clear, simple solution with a bare-bones example.
I'm trying to get the simplest test working with this setup:
.
├── main.py
└── test
├── __init__.py
└── test.py
In my test module, I have the following contents:
test.py
class Test:
def __init__(self):
print('hello')
__init__.py
from test import Test
__all__ = ['Test']
In the upper-level main.py I have the following:
from test import Test
Test()
When I try to run main.py I get:
ImportError: cannot import name 'Test'
I know I could replace the import statement in main.py with from test.test import Test, but my understanding was that one of the points of __init__.py was to make submodules accessible at the package level (with __all__ allowing to import all with from test import *)
According to PEP 404:
In Python 3, implicit relative imports within packages are no longer
available - only absolute imports and explicit relative imports are
supported. In addition, star imports (e.g. from x import *) are only
permitted in module level code.
If you change __init__.py to:
from test.test import Test
__all__ = ['Test']
then your code works:
$ python3 main.py
hello
But now it works only on python3 (and your original code works only on python2).
To have code that works on both lines of python, we have to use explicit relative import:
from .test import Test
__all__ = ['Test']
Code execution:
$ python2 main.py
hello
$ python3 main.py
hello
I have the following file structure
MainDirectory
| Subfolder1
| script1.py
| Subfolder2
| __init__.py
| script2.py
I want to import the the module script2.py in the script1.py.
With Python2.7 I was able to do it like this:
The __init__.py contains the code:
from script2 import ClassA
File script1.py contains the following import structure:
sys.path.insert(0, "../")
from SubFolder2 import ClassA
But when I run the same in Python3 then I get an
ImportError: No module named 'script2'
What would I have to change to get it to work with Python3?
This works as expected:
import sys
import os
sys.path.insert(0, '/'.join(os.path.dirname(os.path.abspath(__file__)).split('/')[0:-1])+'/Subfolder2')
import script2
You need to specify that this resource is in a parent directory...
from ..SubFolder2 import script2