Programmatically import Python files as modules within another Python file and run them - python-3.x

There's a highly upvoted StackOverflow thread which says that the best way to run Python files within another Python file is to import them as a module.
That works well for me, except that I'm having trouble doing it programmatically in a case where there are at least hundreds (if not thousands) of files to be run.
All of the files are in the same directory and share a common naming convention. I tried to run them like this:
import glob, os
for filename in glob.glob("*_decomp*"):
import filename
but that throws an error:
Traceback (most recent call last):
File "C:\Python35\lib\site-packages\IPython\core\interactiveshell.py", line
3066, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 4, in
import filename
File "C:\Program Files\JetBrains\PyCharm Community Edition 2017.1.3\helpers\pydev_pydev_bundle\pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
ImportError: No module named 'filename'
The variable filename is also underlined in red in the IDE, which negates my original hypothesis that it was simply a matter of needing to remove the .py file extension.
This works fine for printing:
import glob, os
for filename in glob.glob("*_decomp*"):
# import filename
print(filename)
So I'm not really sure what the problem with the earlier statement is or how to work around it. I can also do the import manually and that works fine, but again I'd like to do it programmatically so that I don't have to type all of the file names and because the file names will change over time.
Finally, I also tried it with [:-3] (i.e. filename[:-3]) to remove the file extension, but again that only works for print() and not import.

There are other ways of importing not covered by the SO link you give, for example (although I'm not holding this up as a canonical or even necessarily good way of importing, but it works for me) based on one of the examples I found via this SO question/answers Building a minimal plugin architecture in Python I wrote a simple plugin implementation below - it searches a folder called 'plugins' below wherever the .py file with this in it is. Each plugin has to implement a class called Plugin, they all get the same parameters.
path = 'plugins'
# find subdirs of the path - these are the groups
# for each group, load all the .py files, each provides one or more actions
searchdir = os.path.join(os.path.split(__file__)[0],path)
if os.access(searchdir, os.F_OK):
print "searchdir=",searchdir
print "results=",os.walk(searchdir)
(root, dirs, files) =os.walk(searchdir).next()
print root,dirs,files
for dir in dirs:
print "scanning dir",dir
self.groups[dir] = []
sys.path.insert(0, os.path.join(root,dir))
for f in sorted(os.listdir(os.path.join(root,dir))):
print "looking at",f
fname, ext = os.path.splitext(f)
if ext == '.py':
print "importing ",f
mod = __import__(fname)
try:
self.groups[dir].append(mod.PlugIn(group,cmdobj,config, jts_data, directives, current_config_props,allcomponents,globals))
except:
print "URGH! plugin instantiation error!"
raise
sys.path.pop(0)
else:
print "############# no plugins folder",searchdir

Related

Unittest: filename with point not loaded properly

Setup: PyCharm, Python 3.10
We have the naming convention to name our python unittest files like an URL. For example: my.domain.org.py
In the past, this was no issue. Now after an IDE and Python Update it does not run anymore. Selecting right click -> Run "Python tests in my.domain.org.py" throws the error:
Traceback (most recent call last):
File "D:\programs\Python\3.10.2\lib\unittest\loader.py", line 154, in loadTestsFromName
module = __import__(module_name)
ModuleNotFoundError: No module named 'my'
It seems, the loader is interpreting the "." in the filename as path.
How can I run the unittest without renaming the file (which solves the issue)?
you cannot import python files with invalid names (in your case has dots in it) directly, but there's a turn around, you can use the imp library like (here in the example I have a function named print_smth that prints "it works!" in the my.file.py):
import imp
with open('my.file.py', 'rb') as fp:
my_file = imp.load_module(
'my_file', fp, 'my.file.py',
('.py', 'rb', imp.PY_SOURCE)
)
if __name__ == "__main__":
my_file.print_smth()
output:
test.py:1: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
it works!
P.S: preferably DO NOT DO THAT! it is highly deprecated!

Pyinstaller bundled exe not working, but folder exe does [duplicate]

I'm trying to export my .py script to .exe using PyInstaller, which has dependencies on .ui files which were created using Qt Designer.
I can confirm that my .py script works just fine when running it through PyCharm - I'm able to see the GUI I've created with the .ui files.
However, when I export my .py script to .exe and launch it, I recieve the following errors in the command line:
C:\Users\giranm>"C:\Users\giranm\PycharmProjects\PyQt Tutorial\dist\secSearch_demo.exe"
Traceback (most recent call last):
File "secSearch_demo.py", line 13, in <module>
File "site-packages\PyQt4\uic\__init__.py", line 208, in loadUiType
File "site-packages\PyQt4\uic\Compiler\compiler.py", line 140, in compileUi
File "site-packages\PyQt4\uic\uiparser.py", line 974, in parse
File "xml\etree\ElementTree.py", line 1186, in parse
File "xml\etree\ElementTree.py", line 587, in parse
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\giranm\\securitySearchForm.ui'
Failed to execute script secSearch_demo
For some reason, the .exe file is looking for the .ui file within the path - C:\Users\giranm\
However, having done some research already, I was told that I needed to use os.getcwd() and ensure that I have the full path in my script. Even with the code below, I still get errors trying to locate the .ui files.
PyInstaller: IOError: [Errno 2] No such file or directory:
# import relevant modules etc...
cwd = os.getcwd()
securitySearchForm = os.path.join(cwd, "securitySearchForm.ui")
popboxForm = os.path.join(cwd, "popbox.ui")
Ui_MainWindow, QtBaseClass = uic.loadUiType(securitySearchForm)
Ui_PopBox, QtSubClass = uic.loadUiType(popboxForm)
# remainder of code below.
I'm aware that one can convert .ui files to .py and import them into the main routine using pyuic4. However, I will be making multiple edits to the .ui files
and thus it is not feasible for me to keep converting them.
Is there anyway to fix this so that I can create a standalone .exe?
I'm fairly new to using PyQT4 and PyInstaller - any help would be much appreciated!
After scratching my head all weekend and looking further on SO, I managed to compile the standalone .exe as expected using the UI files.
Firstly, I defined the following function using this answer
Bundling data files with PyInstaller (--onefile)
# Define function to import external files when using PyInstaller.
def resource_path(relative_path):
""" Get absolute path to resource, works for dev and for PyInstaller """
try:
# PyInstaller creates a temp folder and stores path in _MEIPASS
base_path = sys._MEIPASS
except Exception:
base_path = os.path.abspath(".")
return os.path.join(base_path, relative_path)
Next I imported the .UI files using this function and variables for the required classes.
# Import .ui forms for the GUI using function resource_path()
securitySearchForm = resource_path("securitySearchForm.ui")
popboxForm = resource_path("popbox.ui")
Ui_MainWindow, QtBaseClass = uic.loadUiType(securitySearchForm)
Ui_PopBox, QtSubClass = uic.loadUiType(popboxForm)
I then had to create a resource file (.qrc) using Qt Designer and embed images/icons using this resource file. Once done, I used pyrcc4 to convert the .qrc file to .py file, which would be imported in the main script.
Terminal
C:\Users\giranm\PycharmProjects\PyQt Tutorial>pyrcc4 -py3 resources.qrc -o resources_rc.py
Python
import resources_rc
Once I have confirmed the main .py script works, I then created a .spec file using PyInstaller.
Terminal
C:\Users\giranm\PycharmProjects\PyQt Tutorial>pyi-makespec --noconsole --onefile secSearch_demo.py
As per PyInstaller's guide, I've added data files by modifying the above .spec file.
https://pythonhosted.org/PyInstaller/spec-files.html#adding-data-files
Finally, I then compiled the .exe using the .spec file from above.
You can simply use:
uic.loadUi(r'E:\Development\Python\your_ui.ui', self)
Use the full path, and use pyinstaller with standard arguments, and it works fine. The r prefix makes sure the backslashes are interpreted literally.
Another method, tested on Ubuntu 20.04 is to add the .ui file to the data section in the spec file. First generate a spec file with pyinstaller --onefile hello.py. Then update the spec file and run pyinstaller hello.spec.
a = Analysis(['hello.py'],
...
datas=[('mainwindow.ui', '.')],
...
The next step is to update the current directory in your Python file. To do this, the os.chdir(sys._MEIPASS) command has to be used. Wrap it in a try-catch for development use when _MEIPASS is not set.
import os
import sys
# Needed for Wayland applications
os.environ["QT_QPA_PLATFORM"] = "xcb"
# Change the current dir to the temporary one created by PyInstaller
try:
os.chdir(sys._MEIPASS)
print(sys._MEIPASS)
except:
pass
from PySide2.QtUiTools import QUiLoader
from PySide2.QtWidgets import QApplication
from PySide2.QtCore import QFile, QIODevice
if __name__ == "__main__":
app = QApplication(sys.argv)
ui_file_name = "mainwindow.ui"
ui_file = QFile(ui_file_name)
if not ui_file.open(QIODevice.ReadOnly):
print(f"Cannot open {ui_file_name}: {ui_file.errorString()}")
sys.exit(-1)
loader = QUiLoader()
window = loader.load(ui_file)
ui_file.close()
if not window:
print(loader.errorString())
sys.exit(-1)
window.show()
sys.exit(app.exec_())

FileNotFoundError: No such file or directory in pygame [duplicate]

This question already has an answer here:
Could not open resource file, pygame error: "FileNotFoundError: No such file or directory."
(1 answer)
Closed 2 years ago.
I am somewhat new to programming, I don't like it that much but as it is school work I have to do it, good thing I find it interesting.
Here is my code, note I'm using VS code (some things are in french but that shouldn't affect anything):
import pygame
screen = pygame.display.set_mode((800,600))
pygame.display.set_caption("Space vaders")
fond = pygame.image.load (testingfield.png)
and I get:
Traceback (most recent call last):
File "c:/Users/faver/Desktop/Space_Invaders-NSI/space_invaders_1.py", line 11, in <module>
fond = pygame.image.load ('testingfield.png')
FileNotFoundError: No such file or directory.
the file is indeed in the folder, and if I put fond = pygame.image.load ((r'C:\Users\faver\Desktop\Space_Invaders-NSI\testingfield.png'))it works perfectly, the problem now is that it'd be hard to actually send it since the other person would lack the folders I place there.
I've also tried import os a = os.path.lexists("/Users/suyeon/Documents/pythonWorkspace/pygame_basic/character.png") print(a) and it returned false, so I'm pretty much confused, I'd appreciate help if possible ^^'
The image file path has to be relative to the current working directory. The working directory is possibly different to the directory of the python file.
The name and path of the file can be get by __file__. The current working directory can be get by os.getcwd() and can be changed by os.chdir(path).
One solution is to change the working directory:
import os
os.chdir(os.path.dirname(os.path.abspath(__file__)))
An alternative solution is to find the absolute path.
If the image is relative to the folder of the python file (or even in the same folder), then you can get the directory of the file and join (os.path.join()) the image filename. e.g.:
import pygame
import os
# get the directory of this file
sourceFileDir = os.path.dirname(os.path.abspath(__file__))
# [...]
# join the filepath and the filename
fondImgPath = os.path.join(sourceFileDir, 'testingfield.png')
fond = pygame.image.load(fondImgPath)

Python Py2exe Error After Compile

I have a pretty complicated application I am writing that integrates a lot of modules. Matplotlib was giving some errors until I explicitly included it in the py2exe file. I am getting this error after I compile and try to launch the program and I don't understand what it means. Any help appreciated!
Py2exe compilation file:
from distutils.core import setup
import py2exe, sys, os
sys.argv.append('py2exe')
setup(
options = {
'py2exe': {
'optimize': 2,
'includes' : ["matplotlib.backends.backend_tkagg"],
'packages' : ['matplotlib', 'pytz'],
}
},
windows = [{'script': "MYAPP.py", "icon_resources": [(1, "s.ico")]}],
zipfile = "shared.lib",
Error when launching after compile:
Traceback (most recent call last):
File "C:\Python34\lib\site-packages\pint\unit.py", line 756, in load_definitions
with closing(pkg_resources.resource_stream(__name__, file)) as fp:
File "C:\Python34\lib\site-packages\pkg_resources\__init__.py", line 1167, in resource_stream
self, resource_name
File "C:\Python34\lib\site-packages\pkg_resources\__init__.py", line 1602, in get_resource_stream
return io.BytesIO(self.get_resource_string(manager, resource_name))
File "C:\Python34\lib\site-packages\pkg_resources\__init__.py", line 1605, in get_resource_string
return self._get(self._fn(self.module_path, resource_name))
File "C:\Python34\lib\site-packages\pkg_resources\__init__.py", line 1683, in _get
return self.loader.get_data(path)
OSError: [Errno 0] Error: 'pint\\default_en.txt'
I think the problem here lies with py2exe's dealing with txt files. It looks like you're using the pint module and within pint is the file default_en.txt (which your compiled app is complaining about). When py2exe compiles pint it ignores the txt files contained within the pint package.
The workaround I used was within my script to state:
ureg = pint.UnitRegistry('\path\to\default_en.txt')
also, I added the following to my setup.py file for py2exe
"packages":["pkg_resources"],
from: http://pint.readthedocs.org/en/0.6/index.html
If you make a translation of the default units or define a completely new set, you don’t want to append the translated definitions so you just give the filename to the constructor:
from pint import UnitRegistry
ureg = UnitRegistry('/your/path/to/default_es.txt')
I made \path\to\ a location within my project and added default_en.txt and constants_en.txt to that folder (copied from pint folder). Then in py2exe I added these files to my data_files.
The real solution would be to figure out how to get py2exe include the txt files in module folders but I haven't yet figured that out.

Cx_Freeze compiled pygame file doesn't work

The following error happens if i try to compile my python (using python 3.2) file:
Traceback(most recent call last):
File
"c:\python32\lib\site-packages\cx_Freeze\initscripts\Console3.py", line
27, in <module>
exec(code, m.__dict__)
File "Abertura.py", line 208, in <module>
File "Abertura.py", line 154, in main
File "Abertura.py", line 9, in __init__
pygame.error: Couldn't open
C:\Python32\build\exe.win32-3.2\library.zip\Imagens\menu1.png
I already included pygame._view and tried to copy the 'Imagens' directory to the library.zip file, but it doesn't work. I'm using images, musics and videos that come's from other directories by including in my code:
def file_path(filename, directory):
return os.path.join(
os.path.dirname(os.path.abspath(__file__)),
directory,
filename
)
And this is my setup.py file:
from cx_Freeze import setup, Executable
exe=Executable(
script="Abertura.py",
base="Win32Gui",
)
includefiles=[('C:\Python32\Imagens', 'Imagens'),
('C:\Python32\Musicas','Musicas'),
('C:\Python32\Videos','Videos')
]
includes=[]
excludes=[]
packages=[]
setup(
version = "1.0",
description = "RPG",
author = "Pedro Forli e Ivan Veronezzi",
name = "Batalha Inapropriada",
options = {'build_exe': {'excludes':excludes,'packages':packages,'include_files':includefiles}},
executables = [exe]
)
How do i fix it?
(sorry about my possibles english mistakes)
Anything accessed from within a compressed archive (such as zips, rars, tar.gzs, etc...) need to be decompressed first before you access them.
That being said, you should not have your recourse files in a zip because decompressing it every time to want to access something is slow, and difficult. Your resource files should be in a normal directory, not an archive.
The reason why you're getting this error is because it's looking for a folder named library.zip and it's not finding one because library.zip is not a folder, it's a file.
How I would suggest to combat this error is to extract everything into a folder named library and to change in your code anywhere that library.zip exists to library.

Resources