I am trying to import a module (not a file) from a different path.
By module, I mean that I have a folder named Util containing a single file named __init__.py.
Using import Util in a python file on the same level as the Util folder works fine.
But this is not the case when the python file resides elsewhere.
Following this answer, I have this piece of code working:
import sys,os
sys.path.append(os.path.dirname(__file__)+'the relative path to the python file that I want to import')
But it works only when I import a python file, not a python module.
How can I resolve this?
Problem solved:
Due to the fact that I am using Windows, the string os.path.dirname(__file__)+'the relative path to the python file that I want to import' contains a mixture of forward slashes (/) and backward slashes (\).
The simple solution is to ensure that the (OS-dependent) expression os.path.dirname(__file__) contains only forward slashes:
os.path.dirname(__file__).replace('\\','/')
Same goes for the constant part 'the relative path to the python file that I want to import' of course, which one can simply fix manually if needed (I already had this one with forward slashes only).
Related
I tried the following in an __init__.py file, thinking that it would be evaluated according to its location at the time of import:
# /../../proj_dir_foo/__init__.py
# opens file: .../proj_dir_foo/foo.csv
import pandas as pd
settings = pd.read_csv('foo.csv')
And from a different file:
from foo.bar.proj_dir_foo import settings
Yields: FileNotFoundError
But this is not really convenient. Instead of accumulating configuration files that are much easier to modify, I am accumulating source code in proj_dir_foo which stores configuration info.
The sole reason it is in source code is because having a project module that knows where the root's resources or materials folder full of configs is is not technically a "module". Instead, it is an integrated cog in a machine. Or, rather, a thing I can no longer easily refactor.
How does one modularize any arbitrary configuration file in python project?
Your script's current directory is the directory from which you started it. import os; print(os.getcwd()) will show you that.
If you want to open a file what sits in a place relative to your code, you have several options.
Use sys.argv[0] to get the path to your script; Use path.dirname() to extract the directory from it, and path.join() make a path to a particular file:
# some script.
import json, sys, path
my_path = path.dirname(sys.argv[0])
cfg_path = path.join(my_path, 'config', 'settings.json')
with open(my_path) as cfg_file:
my_settings = json.load(cfg_file)
Alternatively, if you import a module, you can use its __file__ attribute to learn where did you import it from, and use to locate a config:
# some script.
import path
import foo.bar.baz
baz_cfg_path = path.join(path.dirname(foo.bar.baz.__file__), 'baz.cfg')
I have a project on CDSW organized as follow :
/home/cdsw/my_project_v2.1
|_>input
|_>output
|_>scr
|_>__init__.py
|_>main.py
|_>utils
|_>__init__.py
|_>helpers.py
in my current code, I use sys.path.append to perform my imports.
import sys
sys.path.append("/home/cdsw/my_project_v2.1/src/utils/")
from helpers import bar
This works fine but it is a bad practice because if the version change, then I need to change all my scripts that use the path.
I wanted to replace it with some relative path :
from .utils.helpers import bar
But I got the error :
$ pwd
/home/cdsw
$ python3 my_project_v2.1/src/main.py
Traceback (most recent call last):
File "my_project_v2.1/src/main.py", line 1, in <module>
from .utils.helpers import bar
ModuleNotFoundError: No module named '__main__.helpers'; '__main__' is not a package
what do I need to change in my architecture or in my code to make it work ?
Just use
from utils.helpers import bar
A short excerpt from the documentation of the Python command line arguments:
If the script name refers directly to a Python file, the directory containing that file is added to the start of sys.path, and the file is executed as the __main__ module.
The first half of the sentence means that you can use absolute module names when referring to the contents of your directory, because Python will search for module there. The fact that you can not use relative imports is a consequence of the second half of the sentence.
As a side note, you may also consider omitting the version number from the name of the directory, or better yet, put your code directly in /home/cdsw. The latter may sound strange as you would never do that on a regular machine, but here everything is in a container and actually this is the way your code is supposed to be organized in CDSW. You can confirm this by creating a new project based on a template or a git URL – both will put code directly in the home directory.
So I builded a python package localy:
cgi#cgires:~$ pip list | grep mads
madscgi 0.1.0
Its nice! Afterwards I can use it in Jupyter Notebook, in iPython Shell, in Python Shell and even in python scripts outside the modules code. So it works as expected 100% outside the modules code:
Thats nice, but next I want to import code from one builded module (inside the package) into another python file (inside the package). Lets name it import_test.py and try it out:
So it fails if it is getting executed in the directory, where the package is build from. And it looks like, that the python interpreter is taking the parent directory (with the same name like the module) and this is failing.
Is is possible to enforce the usage of the installed pip-package?
As #MisterMiyagi pointed out, the problem was, that there were an upper folder which had the same name as the module.
Here: mads_cons is the upper folder from import_test.py. Therefore, the upper folder is getting imported instead of the via pip installed module. Thats it.
The file you want to import should either be in the same folder or referred to with the absolute path of it.
If that doesn't suit you, you can call sys.path
import sys
sys.path
You can keep your file in any of the directories sys.path returns.
Smart would be, if you keep the file inside.
......../site-packages/
My input is a list of file names (as a list of variables) and I know the path to these files. Each file has a function called "test" and I have to call the "test" function from each of these files. The path is not my working directory. I need to be able to dynamically import these files.
I tried using importlib, but I get the following errors:
import importlib
importlib.import_module("..\..\foo", package=None)
TypeError: the 'package' argument is required to perform a relative import for '..\\..\\x0coo'
importlib.import_module("C:\Users\Desktop\foo", package=None)
ModuleNotFoundError: No module named 'C:\\Users\\Desktop\\foo'
How would I go about executing the function in a file, using the filename and path (both stored in variables)?
I'm not sure this is the best way, but I solved this by first adding the path to
the module to sys.path:
>>import sys
>>sys.path.append('/path/to/module')
>>mod=importlib.import_module('my_module_name')
then you can call functions in that module like this
>>mod.myfunc(myargs)
or, if you have the function name in a python string like func='myfunctionname'
you can call it like
>>mod.__dict__[func](args)
I'd need to see more code, but my first guess is that you need to replace \ with / in your Directory string, since \ escapes out of the string.
This is my directory structure
-directory1
|-currentlyWritingCode
-directory2
|-__init__.py
|-helper.py
|-view.ui
The helper.py is a UI from pyqt4 framework. It needs the view.ui file .
it has following code to load the ui data
viewBase, viewForm = uic.loadUiType("view.ui")
now in directory1 in the currently writing code I did this
import directory2.helper
when I run the code in currentlyWritingCode it is throwing an error that
FileNotFoundError: [Errno 2] No such file or directory: 'view.ui'
What to do now?
using python3.5 from anaconda on Centos 7
Use os.path.join(os.path.dirname(os.path.realpath(__file__)),'view.ui') in place of view.ui. This will ensure you correctly reference the folder that the python file lives in, regardless of the code that imports it.
Note: Make sure you have import os with your other imports.
How does this work?
__file__ is an attribute of the module you are importing. It contains the path to the module file. See this answer. However, this path is not necessarily an absolute path. os.path.realpath returns the absolute path (it even follows symlinks if there are any). At this point we have a full path to the module, so we take the path to the directory (os.path.dirname) and join it with the original filename (which we assumed to be relative to the original module and so should be in the aforementioned directory). os.path.join ensures that the correct \ or / is used when constructing a file path so that the code works on any platform.