My input is a list of file names (as a list of variables) and I know the path to these files. Each file has a function called "test" and I have to call the "test" function from each of these files. The path is not my working directory. I need to be able to dynamically import these files.
I tried using importlib, but I get the following errors:
import importlib
importlib.import_module("..\..\foo", package=None)
TypeError: the 'package' argument is required to perform a relative import for '..\\..\\x0coo'
importlib.import_module("C:\Users\Desktop\foo", package=None)
ModuleNotFoundError: No module named 'C:\\Users\\Desktop\\foo'
How would I go about executing the function in a file, using the filename and path (both stored in variables)?
I'm not sure this is the best way, but I solved this by first adding the path to
the module to sys.path:
>>import sys
>>sys.path.append('/path/to/module')
>>mod=importlib.import_module('my_module_name')
then you can call functions in that module like this
>>mod.myfunc(myargs)
or, if you have the function name in a python string like func='myfunctionname'
you can call it like
>>mod.__dict__[func](args)
I'd need to see more code, but my first guess is that you need to replace \ with / in your Directory string, since \ escapes out of the string.
Related
I tried the following in an __init__.py file, thinking that it would be evaluated according to its location at the time of import:
# /../../proj_dir_foo/__init__.py
# opens file: .../proj_dir_foo/foo.csv
import pandas as pd
settings = pd.read_csv('foo.csv')
And from a different file:
from foo.bar.proj_dir_foo import settings
Yields: FileNotFoundError
But this is not really convenient. Instead of accumulating configuration files that are much easier to modify, I am accumulating source code in proj_dir_foo which stores configuration info.
The sole reason it is in source code is because having a project module that knows where the root's resources or materials folder full of configs is is not technically a "module". Instead, it is an integrated cog in a machine. Or, rather, a thing I can no longer easily refactor.
How does one modularize any arbitrary configuration file in python project?
Your script's current directory is the directory from which you started it. import os; print(os.getcwd()) will show you that.
If you want to open a file what sits in a place relative to your code, you have several options.
Use sys.argv[0] to get the path to your script; Use path.dirname() to extract the directory from it, and path.join() make a path to a particular file:
# some script.
import json, sys, path
my_path = path.dirname(sys.argv[0])
cfg_path = path.join(my_path, 'config', 'settings.json')
with open(my_path) as cfg_file:
my_settings = json.load(cfg_file)
Alternatively, if you import a module, you can use its __file__ attribute to learn where did you import it from, and use to locate a config:
# some script.
import path
import foo.bar.baz
baz_cfg_path = path.join(path.dirname(foo.bar.baz.__file__), 'baz.cfg')
I am trying to import a module (not a file) from a different path.
By module, I mean that I have a folder named Util containing a single file named __init__.py.
Using import Util in a python file on the same level as the Util folder works fine.
But this is not the case when the python file resides elsewhere.
Following this answer, I have this piece of code working:
import sys,os
sys.path.append(os.path.dirname(__file__)+'the relative path to the python file that I want to import')
But it works only when I import a python file, not a python module.
How can I resolve this?
Problem solved:
Due to the fact that I am using Windows, the string os.path.dirname(__file__)+'the relative path to the python file that I want to import' contains a mixture of forward slashes (/) and backward slashes (\).
The simple solution is to ensure that the (OS-dependent) expression os.path.dirname(__file__) contains only forward slashes:
os.path.dirname(__file__).replace('\\','/')
Same goes for the constant part 'the relative path to the python file that I want to import' of course, which one can simply fix manually if needed (I already had this one with forward slashes only).
I have a project on CDSW organized as follow :
/home/cdsw/my_project_v2.1
|_>input
|_>output
|_>scr
|_>__init__.py
|_>main.py
|_>utils
|_>__init__.py
|_>helpers.py
in my current code, I use sys.path.append to perform my imports.
import sys
sys.path.append("/home/cdsw/my_project_v2.1/src/utils/")
from helpers import bar
This works fine but it is a bad practice because if the version change, then I need to change all my scripts that use the path.
I wanted to replace it with some relative path :
from .utils.helpers import bar
But I got the error :
$ pwd
/home/cdsw
$ python3 my_project_v2.1/src/main.py
Traceback (most recent call last):
File "my_project_v2.1/src/main.py", line 1, in <module>
from .utils.helpers import bar
ModuleNotFoundError: No module named '__main__.helpers'; '__main__' is not a package
what do I need to change in my architecture or in my code to make it work ?
Just use
from utils.helpers import bar
A short excerpt from the documentation of the Python command line arguments:
If the script name refers directly to a Python file, the directory containing that file is added to the start of sys.path, and the file is executed as the __main__ module.
The first half of the sentence means that you can use absolute module names when referring to the contents of your directory, because Python will search for module there. The fact that you can not use relative imports is a consequence of the second half of the sentence.
As a side note, you may also consider omitting the version number from the name of the directory, or better yet, put your code directly in /home/cdsw. The latter may sound strange as you would never do that on a regular machine, but here everything is in a container and actually this is the way your code is supposed to be organized in CDSW. You can confirm this by creating a new project based on a template or a git URL – both will put code directly in the home directory.
This is my directory structure
-directory1
|-currentlyWritingCode
-directory2
|-__init__.py
|-helper.py
|-view.ui
The helper.py is a UI from pyqt4 framework. It needs the view.ui file .
it has following code to load the ui data
viewBase, viewForm = uic.loadUiType("view.ui")
now in directory1 in the currently writing code I did this
import directory2.helper
when I run the code in currentlyWritingCode it is throwing an error that
FileNotFoundError: [Errno 2] No such file or directory: 'view.ui'
What to do now?
using python3.5 from anaconda on Centos 7
Use os.path.join(os.path.dirname(os.path.realpath(__file__)),'view.ui') in place of view.ui. This will ensure you correctly reference the folder that the python file lives in, regardless of the code that imports it.
Note: Make sure you have import os with your other imports.
How does this work?
__file__ is an attribute of the module you are importing. It contains the path to the module file. See this answer. However, this path is not necessarily an absolute path. os.path.realpath returns the absolute path (it even follows symlinks if there are any). At this point we have a full path to the module, so we take the path to the directory (os.path.dirname) and join it with the original filename (which we assumed to be relative to the original module and so should be in the aforementioned directory). os.path.join ensures that the correct \ or / is used when constructing a file path so that the code works on any platform.
What I mean is if there's some standard name that can be used for the primary file in a local Nim module so that when importing by path, we can simply reference the directory?
For example, it seems right now I need to specify both the directory and file name of the local module like this:
import my_module.main
Whereas I was hoping to simply be able to reference the directory if the expected file name was found:
import my_module
Aside from using Nimble or creating a separate --path flag for every module in a nim.cfg file, is there anything that will allow this?
Also, in general, is there a conventional name to use for the main file of a module?
If I have a simple app like this:
myapp--+
|
+--myapp.nim
|
+--sub--+
|
+--sub.nim
In myapp.nim I have to use import sub.sub to load the sub-module. With import sub, I get Error: cannot open sub.
Likewise if I have modules outside the app and set the --path to the parent of those modules, I have to use import my_module.my_module.
The idea is to have a my_module.nim and sub-modules in a my_module directory. Then you can just import my_module.