I've read PEP 328 and similar questions like this, but they still didn't answer my questions about python 3 import. My question is:
why does a single dot . relative import ever produce an error like the following?
code: (__init__.py exists in the same directory)
from . import module_x
error:
ImportError: attempted relative import with no known parent package
Yes, there are workarounds to get around of this error for example by running python -m from the parent directory, but why?
Shouldn't . always refer to the same directory of the current module and the example always works as long as the module_x exists in the same directory?
Particularly PyCharm tries to run unittest test cases from the directory where the test file is, and would always fail the . import. Some may argue it's PyCharm's problem, but I think a simple import like this should really always work regardless where the interpreter runs.
There is a detailed answer here, also here. Sorry I cannot write this as a comment.
Related
I'm trying out Freeling's API for python. The installation and test were ok, they provide a sample.py file that works perfectly (I've played around a little bit with it and it works).
So I was trying to use it on some other python code I have, in a different folder (I'm kind of guessing this is a path issue), but whenever I import freeling (like it shows on the sample.py):
import freeling
FREELINGDIR = "/usr/local";
DATA = FREELINGDIR+"/share/freeling/";
LANG="es";
freeling.util_init_locale("default");
I get this error:
ModuleNotFoundError: No module named 'freeling'.
The sample.py is located on the ~/Freeling-4.0/APIs/Python/ folder, while my other file is located in ~/project/, I dont know if that can be an issue.
Thank you!
A simple solution is to have a copy of freeling.py in the same directory as your code, since python will look there.
A better solution is to either paste it in one of the locations where it usually checks (like the lib folder in its install directory), or to tell it that the path where your file is should be scanned for a module.
You can check out this question to see how it can be done on Windows. You are basically just setting the PYTHONPATH environment variable, and there will only be minor differences in how to do so for other OSes. This page gives instructions that should work on Linux systems.
I like this answer since it adds the path at runtime in the script itself, doesn't make persistent changes, and is largely independent of the underlying OS (apart from the fact that you need to use the appropriate module path of course).
You need to set PYTHONPATH so python can find the modules if they are not in the same folder.
I've been working on a python3 application and I ran into a strange problem that picked my curiosity after annoying me greatly.
My file structure is something like this:
root/
| __init__.py
| main.py
| fuzzy/
| __init__.py
| foo.py
| dreamy/
| __init__.py
| bar.py
| meh.py
So I need to use a method of the foo module in both bar and meh.
In bar I wrote:
from fuzzy.foo import foo_function
And that worked perfectly fine.
Now in meh I used the exact same syntax but ended up with an import error:
ModuleNotFoundError: No module named 'fuzzy'
I managed to solve the problem using this method, but I would like to understand why this happens.
According to the Python3 documentation my import syntax is correct:
An alternative way of importing the submodule is:
from sound.effects import echo
This seems to also be the syntax provided by answered question on this forum
Moreover, both bar and meh are in the same directory and their code starts in exactly the same way.
Does anyone know why this error occurs in one file but not in the other ?
Is there something I did wrong ?
If I did do something wrong, what would be the correct way (or good practices) to import local packages / methods ?
Thanks for your insight.
---EDIT---
I did not fiddle with my PYTHONPATH in any way, and I am running Python3 from a (vanilla) Conda venv. For good measures, I also created a new file structure and copied the code in new, blank files. This was to make sure that nothing "funny" would have happened to my directory.
This question seems to be terribly simple, however I did notice that other people struggled with it.
Though I found the answer to my question thanks to Baum mit Augen.
Baum mit Augen took the time to write this answer which is very instructive and well written.
So sum my problem up:
In development, I tried to call my functions by launching my bar.py from the terminal. This means bar is the main, but located inside a package subdirectory. This causes an Import Error because "its name does not reveal that it is in a package".
I just want to highlight 2 things unrelated to my question:
Considering the post on which I have found this answer this is NOT an easy problem. FYI: I had a quick chat with a Python developer who couldn't find the words to explain this to me.
I find it incredible that those who seek help with valid questions are basically being answered to "RTFM", arch-linux style. I really find it a pity.
Suppose we have the following project structure:
E:\demo_proj\
utilities_package\
__init__.py
util_one.py
util_two.py
demo_package\
__init__.py
demo_sub_package\
__init__.py
demo_sub_sub_package\
demo_file.py
What is a sensible way for demo_file.py to import from utilities_package?
The utilities_package is in a much higher level directory than demo_file.py.
Although it is not shown in the little diagram I gave above, suppose that utilities_package is used everywhere throughout the project. It is not sensible to make utilities_package a sub-directory of demo_sub_sub_package
Some similar questions have been asked here and here.
The asker of the first question neglected to include a __init__.py in their packages. That is not an issue in my case.
The answers given to the the second question linked-to above are problematic:
Most of the answers are quite old. For example, Loader.load_module was deprecated after Python 3.4
Most of the answers are specific to the question in that they only go up one directory, whereas, I would like to go up several levels.
Some of the answers are high-level descriptions without code. Other answers have the opposite problem: they contain ugly-looking code, but don't bother to explain it.
I am open to solutions which require restructuring the project. I'm also interested in solutions which leave the project structure intact, but which insert the relevant code into demo_file.py
It already has the very epic solution for your problems. Please refer there: Relative imports for the billionth time. Just as a notation, the python3 and python2 have the totally different definitions for the relative and absolute import. For version 3+, any modules import directly using import module_name are treated as the absolute importation, which means this module must be included in the sys.path. Otherwise, for any modules import like from one_module import something or from . import something are treated like the relative importation. The dot-notation represent the current directory. However, any module is executed directly by itself, it will lose its namespace, instead it will always be treated as __main__, and all the files in the same directory will automatically add to sys.path by default. Hope it will help you.
I'll first begin by stating that I have been coming back to this problem over and over again for the last several months. No matter how much I research into it, I cannot seem to find a satisfactory answer. I'm hoping that the community here can help me out.
Basic problem - consider this structure of python packages and python modules.
|- src
|- pkg_1
|- foo_1.py
|- foo_2.py
|- __init__.py
|- pkg2
|- bar_1.py
|- bar_2.py
|- __init__.py
|- do_stuff.py
|- __init__.py
Suppose that module bar_2 needs to import from module foo_1. My options are myriad, but a few present themselves quickly.
(My preferred way) Module bar_2 can do from ..pkg_1 import foo_1. This is great because it doesn't require what amounts to hard-coding a path into the module, thereby allowing flexibility, ease of maintenance, all that. In do_stuff.py if I then write from src.pkg_2 import bar_2 and then run, I am golden. Here is an example setup:
foo_1.py:
class Foo_1():
def __init__(self):
print('Hello from foo_1!')
bar_2.py:
from ..pkg_1 import foo_1
class Bar_2():
def __init__(self):
print('Hello from bar_2!')
foo_1.Foo_1() #Prints foo_1 message!
do_stuff.py:
from src.pkg_2 import bar_2
bar_2.Bar_2()
Console prints:
Hello from bar_2!
Hello from foo_1!
All is well. However, consider the next scenario.
Suppose now that I want to run bar_2 as __main__, as follows:
from ..pkg_1 import foo_1
class Bar_2():
def __init__(self):
print('Hello from bar_2!')
foo_1.Foo_1()
if __name__ == '__main__':
Bar_2()
A SystemError is raised: from ..pkg_1 import foo_1
SystemError: Parent module '' not loaded, cannot perform relative import
For far longer than I care to admit, I did not understand the reason for this. The solution, though, lies in the fact that when you run a module directly, its __name__ variable is set to __main__. Since the relative imports establish position in the hierarchy with __name__, this means that there is no directory information to parse to figure things out. This makes loads of sense, and I feel very dumb for having not realized it before.
So, thus began my quest (yeah, just getting started). Eventually I learned of the __package__ variable. Reading about it in the PEP notes, it seemed as though it would solve all my problems! So I tried the following boilerplate code before the import statements in bar_2:
if __name__ == '__main__':
__package__ = 'src.pkg_2'
This did not work. :(
I since have come to find out that Guido has addressed this very issue and that he regards the whole notion of running a module from within a package as anti-pattern.
See this link:
https://mail.python.org/pipermail/python-3000/2007-April/006793.html
This makes sense, as I will be the first to admit that I only do it for on the fly testing... which should never be done! (Right??) Therefore, as best as I understand, there are no elegant ways to run the module from within a package UNLESS you do absolute imports... which I would like to avoid.
So, after all of that, here is my question: Should I use one of the many hacky ways out there to get around this problem and do unholy things with the system path so that I can have my relative import cake and eat it (i.e. test on the fly by running as __main__) too?? I think I already know the answer (I just would like some wizened Yoda-like person to confirm).
Possible Answer:
Use relative imports, because hard-coding paths (and hard-coding in general if avoidable) is ant-pattern.
Do not bother with running modules nested in packages as __main__... instead, run them from your testing module (which you wrote first... right??).
Thank you for taking the time to read this question. I realize there are many other questions regarding this topic... but I wanted to 'share my journey' and see if my current course of action is correct.
Running the following in src/:
python -m pkg2.bar_2
will have pkg2/bar_2.py be your main script while still being inside a package, meaning that relative imports will work.
I follow these rules which prevent any issues:
Always use full imports, no relative imports.
Always start a Python process from the root of the project or using the full absolute path.
It's rather rare that you benefit from relative imports when you have to move files around (and when you do, it's not that much work to rename a few import lines).
Full paths from the root of a project or using the full path on disk of a file removes any ambiguity and will run the desired file.
My answer (and that of many CPython core developers) is essentially the same as Simeon's. The only thing not hard-coded by relative imports is the name of the package ('src', in this case). But you hard-coded it in do_stuff.py anyway. Within-subpackage relative imports (unlike the cross-subpackage imports you exhibit) let you copy the subpackage to another package (with a different name), and possibly change the subpackage name as you do so. Do you really need that particular flexibility? Is it really worth more than the very real cost of relative imports?
Why the subpackages, instead of putting everything in the main package, src? Idlelib has about 60 run-time modules in idlelib/ itself. The only subpackage is idle_test, for test modules. All imports start with idlelib.
I am a big fan of being able to run the test for one module (rather than the package test suite) by running non-cli modules as the main module. It encourages incremental TDD. So I have 'if name.. ' clauses in both run-time and test modules in idlelib.
I'm trying to get the pymysql module working with python3 on a Macintosh. Note that I am a beginning python user who decided to switch from ruby and am trying to build a simple (sigh) database project to drive my learning python.
In a simple (I thought) test program, I am getting a syntax error in confiparser.py (which is used by the pymysql module)
def __init__(self, defaults=None, dict_type=_default_dict,
allow_no_value=False, *, delimiters=('=', ':'),
comment_prefixes=('#', ';'), inline_comment_prefixes=None,
strict=True, empty_lines_in_values=True,
default_section=DEFAULTSECT,
interpolation=_UNSET):
According to Komodo, the error is on the second line. I assume it is related to the asterix but regardless, I don't know why there would be a problem like this with a standard Python module.
Anyone seen this before?
You're most certainly running the code with a 2.x interpreter. I wonder why it even tries to import 3.x libraries, perhaps the answer lies in your installation process - but that's a different question. Anyway, this (before any other imports)
import sys
print(sys.version)
should show which Python version is actually run, as Komodo Edit may be choosing the wrong executable for whatever reason. Alternatively, leave out the parens and it simply fails if run with Python 3.
In Python 3.2 the configparser module does indeed look that way. Importing it works fine from Python 3.2, but not from Python 2.
Am I right in guessing you get the error when you try to run your module with Komodo? Then you just have configured the wrong Python executable.