How do I load a python module, that is not built in. I'm trying to create a plugin system for a small project im working on. How do I load those "plugins" into python? And, instaed of calling "import module", use a string to reference the module.
Have a look at importlib
Option 1: Import an arbitrary file in an arbiatrary path
Assume there's a module at /path/to/my/custom/module.py containing the following contents:
# /path/to/my/custom/module.py
test_var = 'hello'
def test_func():
print(test_var)
We can import this module using the following code:
import importlib.machinery
myfile = '/path/to/my/custom/module.py'
sfl = importlib.machinery.SourceFileLoader('mymod', myfile)
mymod = sfl.load_module()
The module is imported and assigned to the variable mymod. We can then access the module's contents as:
mymod.test_var
# prints 'hello' to the console
mymod.test_func()
# also prints 'hello' to the console
Option 2: Import a module from a package
Use importlib.import_module
For example, if you want to import settings from a settings.py file in your application root folder, you could use
_settings = importlib.import_module('settings')
The popular task queue package Celery uses this a lot, rather than giving you code examples here, please check out their git repository
Related
I have the following file structure
home/user/app.py
home/user/content/resource.py
home/user/content/call1.py
home/user/content/call2.py
I have imported resources.py in app.py as below:
import content.resource
Also, I have imported call1 and call2 in resource.py
import call1
import call2
The requirement is to run two tests individually.
run app.py
run resource.py
When I run app.py, it says cannot find call1 and call2.
When run resource.py, the file is running without any issues. How to run app.py python file to call import functions in resource.py and also call1.py and call2.py files?
All the 4 files having __init__ main function.
In your __init__ files, just create a list like this for each init, so for your user __init__: __all__ = ["app", "content"]
And for your content __init__: __all__ = ["resource", "call1", "call2"]
First try: export PYTHONPATH=/home/user<-- Make sure this is the correct absolute path.
If that doesn't solve the issue, try adding content to the path as well.
try: export PYTHONPATH=/home/user/:/home/user/content/
This should definitely work.
You will then import like so:
import user.app
import user.content.resource
NOTE
Whatever you want to use, you must import in every file. Don't bother importing in __init__. Just mention whatever modules that __init__ includes by doing __all__ = []
You have to import call1 and call2 in app.py if you want to call them there.
My goal is to define a function that gets a relative path and returns an absolute path.
The issue is that I want this function to be located in a specific module and that other modules will be able to use it.
If the function would be located in each module (and not being imported) it would have looked something like this:
import os
def get_absolute(relative):
dir_path = os.path.dirname(os.path.realpath(__file__))
return os.path.abspath(os.path.join(dir_path, relative_path))
absolute = get_absolute('../file.txt')
but if I want to define this function once (in one module) and be able to import it by other modules I will need to add another argument to the function for the directory path of the module that imported the function like so:
helper.py
import os
def get_absolute(relative, dir_path):
return os.path.abspath(os.path.join(dir_path, relative_path))
importer.py
import os
from helper import get_absolute
DIR_PATH = os.path.dirname(os.path.realpath(__file__))
absolute = get_absolute('../file.txt', DIR_PATH)
I saw Get path of importing module but in my case the module is unknown (multiple modules can import this function).
Is there a way for the helper module to know the directory path of the importing module without passing it as an argument?
I have a main config.py file and then specific client config files, e.g. client1_config.py.
What I'd like to do is import all variables within my client1_config.py file into my config.py file. The catch is I want to do this flexibly at runtime according to an environment variable. It would look something like this:
import os
import importlib
client = os.environ['CLIENT']
client_config = importlib.import_module(
'{client}_config'.format(
client=client))
from client_config import *
This code snippet returns the following error: ModuleNotFoundError: No module named 'client_config'
Is it possible (and how) to achieve what I'm trying to do or Python does not support this kind of importing at all?
The call to import_module already imports the client configuration. from client_config import * assumes that client_config is the name of the module you are trying to import, just as import os will import the module os even if you create a variable os beforehand:
os = "sys"
import os # still imports the os module, not the sys module
In the following, assume that we have a client1_config.py which just contains one variable:
dummy = True
To add its elements to the main namespace of config.py so that you can access them directly, you can do the following:
import importlib
client = "client1"
# Import the client's configuration
client_config = importlib.import_module(f"{client}_config")
print(client_config.dummy) # True
# Add all elements from client_config
# to the main namespace:
globals().update({v: getattr(client_config, v)
for v in client_config.__dict__
if not v.startswith("_")})
print(dummy) # True
However, I would suggest to access the client's configuration as config.client for clarity and to avoid the client's configuration file overwriting values in the main configuration file.
I have a file structure as shown below,
MainFolder
__init__.py
FirstFolder
__init__.py
firstFile.py
SecondFolder
__init__.py
secondFile.py
Inside firstFile.py, I have a class named Math and I want to import this class in secondFile.py.
Code for firstFile.py
class Math(object):
def __init__(self, first_value, second_value):
self.first_value = first_value
self.second_value = second_value
def addition(self):
self.total_add_value = self.first_value + self.second_value
print(self.total_add_value)
def subtraction(self):
self.total_sub_value = self.first_value - self.second_value
print(self.total_sub_value)
Code for secondFile.py
from FirstFolder.firstFile import Math
Math(10, 2).addition()
Math(10, 2).subtraction()
When I tried running secondFile.py I get this error: ModuleNotFoundError: No module named 'First'
I am using Windows and the MainFolder is located in my C drive, under C:\Users\Name\Documents\Python\MainFolder
Possible solutions that I have tried are, creating the empty __init__.py for all main and sub folders, adding the dir of MainFolder into path under System Properties environment variable and using import sys & sys.path.append('\Users\Name\Documents\Python\MainFolder').
Unfortunately, all these solutions that I have found are not working. If anyone can highlight my mistakes to me or suggest other solutions, that would be great. Any help will be greatly appreciated!
There are potentially two issues. The first is with your import statement. The import statement should be
from FirstFolder.firstFile import Math
The second is likely that your PYTHONPATH environment variable doesn't include your MainFolder.
On linux and unix based systems you can do this temporarily on the commandline with
export PYTHONPATH=$PYTHONPATH:/path/to/MainFolder
On windows
set PYTHONPATH="%path%;C:\path\to\MainFolder"
If you want to set it permanently, use setx instead of set
I found a solution to add files in library.zip via: Extend py2exe to copy files to the zipfile where pkg_resources can load them.
I can access to my file when library.zip is not include the exe.
I add a file : text.txt in directory: foo/media in library.zip.
And I use this code:
import pkg_resources
import zipfile
from cStringIO import StringIO
my_data = pkg_resources.resource_string(__name__,"library.zip")
filezip = StringIO(my_data)
zip = zipfile.ZipFile(filezip)
data = zip.read("foo/media/text.txt")
I try to use pkg_resources but I think that I don't understand something because I could open directly "library.zip".
My question is how can I do this when library.zip is embed in exe?
Best Regards
Jean-Michel
I cobbled together a reasonably neat solution to this, but it doesn't use pkg_resources.
I need to distribute productivity tools as standalone EXEs, that is, all bundled into the one .exe file. I also need to send out notifications when these tools are used, which I do via the Logging API, using file-based configuration. I emded the logging.cfg fileto make it harder to effectively switch-off these notifications i.e. by deleting the loose file... which would probably break the app anyway.
So the following is the interesting bits from my setup.py:
LOGGING_CFG = open('main/resources/logging.cfg').read()
setup(
name='productivity-tool',
...
# py2exe extras
console=[{'script': productivity_tool.__file__.replace('.pyc', '.py'),
'other_resources': [(u'LOGGINGCFG', 1, LOGGING_CFG)]}],
zipfile=None,
options={'py2exe': {'bundle_files': 1, 'dll_excludes': ['w9xpopen.exe']}},
)
Then in the startup code for productivity_tool.py:
from win32api import LoadResource
from StringIO import StringIO
from logging.config import fileConfig
...
if __name__ == '__main__':
if is_exe():
logging_cfg = StringIO(LoadResource(0, u'LOGGINGCFG', 1))
else:
logging_cfg = 'main/resources/logging.cfg'
fileConfig(logging_cfg)
...
Works a treat!!!
Thank you but I found the solution
my_data = pkg_resources.resource_stream("__main__",sys.executable) # get lib.zip file
zip = zipfile.ZipFile(my_data)
data = zip.read("foo/media/doc.pdf") # get my data on lib.zip
file = open(output_name, 'wb')
file.write(data) # write it on a file
file.close()
Best Regards
You shouldn't be using pkg_resources to retrieve the library.zip file. You should use it to retrieve the added resource.
Suppose you have the following project structure:
setup.py
foo/
__init__.py
bar.py
media/
image.jpg
You would use resource_string (or, preferably, resource_stream) to access image.jpg:
img = pkg_resources.resource_string(__name__, 'media/image.jpg')
That should "just work". At least it did when I bundled my media files in the EXE. (Sorry, I've since left the company where I was using py2exe, so don't have a working example to draw on.)
You could also try using pkg_resources.resource_filename(), but I don't think that works under py2exe.