I've recently been running into a Import Error from Shiboken2 that I didn't have before with my executable. I have an application that I built in PyCharm and I build an .exe for it with CX_Freeze. I have tried every single latest version of Python but I am 100% confident now that it's not the version of Python I have that is the issue. Especially, since the app run's perfectly when I run the source code, but when I use my Setup.py script to build it, I keep getting the following issue when I try to run it:
Here is how my Setup.py script looks:
import sys
import os
from cx_Freeze import setup, Executable
sys.path.append(os.path.abspath("./src/"))
sys.path.append(os.path.abspath("./src/gui/rc/"))
sys.path.append(os.path.abspath("./database/component_actions"))
# Dependencies are automatically detected, but it might need
# fine tuning.
buildOptions = {
"packages": [
# Facile sub-packages
"src.gui",
"src.qt_models",
"src.data",
"src.libs",
"src.tguiil",
"src.graphics",
"src.tools",
],
"includes": ["scipy.sparse.csgraph._validation",
"scipy.ndimage._ni_support",
"scipy._distributor_init",
],
"include_files": ["database/",
"src/tguiil/",
"src/data/"
],
"excludes": ["scipy.spatial.cKDTree",
]
}
installOptions = {"skip_build":True}
base = None
# Uncomment for GUI applications to NOT show cmd window while running.
if sys.platform =='win32':
base = 'Win32GUI'
executables = [
Executable(script = 'src/facile.py', base=base, targetName = 'facile.exe', icon = 'resources/facade_logo_256.ico')
]
setup(name='***',
version = '***',
description = '***',
options = {
"build_exe": buildOptions,
"install_exe": installOptions,
},
executables = executables)
(Sorry there's some info I am trying to remain hidden so that's why I use ***)
When the Problem started:
I updated Shiboken 2 to version 5.15.0 but I was aware that I needed to update Pyside2 so I don't understand how this updated packages effects my executable build.
If need be I can show all my dependencies but I don't think that's the issue because as I said before the source code works properly when I run it on PyCharm, only when I build the executable and try to run it do I have this issue. Maybe this is a PATH issue? Or some dependency I need to include in the setup.py?
B.T.W. I am using a Virtual Environment with Python 3.7.4 32-bit as the interpreter.
I believe this issue stemmed from my environment. I don't really have an adamant answer because I did multitude of things such as the following in this order:
Downloaded PyCharm 2020 and deleted PyCharm 2019.
Deleted all my past Python packages/executables/dependencies, deleted my old virtual environment, and old build folder.
Reinstalled Python 3.7.4 and checked the PATH tab.
Added a Python Interpreter with the Python 3.7.4 in virtual environment through PyCharm 2020's IDE.
Pip installed all necessary dependencies from my requirements.txt.
I made sure to give PyCharm enough time to think (load) in between each step and once I was finished I built the executable and it ran correctly.
I hope this helps anyone who runs into a similar problem.
Related
How do I setup a Python virtual environment with the FreeCAD library embedded as to enable import as a module into scripts?
I would like to avoid using the FreeCAD GUI as well as being dependent on having FreeCAD installed on the system, when working with Python scripts that use FreeCAD libraries to create and modify 3D geometry. I hope a Python virtual environment can make that possible.
I am working in PyCharm 2021.1.1 with Python 3.8 in virtualenv on Debian 10.
I started out with FreeCAD documentation for embedding in scripts as a basis:
https://wiki.freecadweb.org/Embedding_FreeCAD
In one attempt, I downloaded and unpacked .deb packages from Debian, taking care to get the correct versions required by each dependency. In another attempt, I copied the contents of a FreeCAD flatpak install, as it should contain all the libraries that FreeCAD depends on.
After placing the libraries to be imported in the virtual maching folder, I have pointed to them with sys.path.append() as well as PyCharm's Project Structure tool in various attempts. In both cases the virtual environment detects where FreeCAD.so is located, but fails to find any of its dependencies, even when located in the same folder. When importing these dependencies explicitly, each of them have the same issue. This leads to a dead end when an import fails because it does not define a module export function according to Python:
ImportError: dynamic module does not define module export function (PyInit_libnghttp2)
I seem to be looking at a very long chain of broken dependencies, even though I make the required libraries available and inform Python where they are located.
I would appreciate either straight up instructions for how to do this or pointers to documentation that describes importing FreeCAD libraries in Python virtual environments, as I have not come across anything that specific yet.
I came across a few prior questions which seemed to have similar intent, but no answers:
Embedding FreeCAD in python script
Is it possible to embed Blender/Freecad in a python program?
Similar questions for Conda focus on importing libraries from the host system rather than embedding them in the virtual environment:
Incude FreeCAD in system path for just one conda virtual environment
Other people's questions on FreeCAD forums went unanswered:
https://forum.freecadweb.org/viewtopic.php?t=27929
EDIT:
Figuring this out was a great learning experience. The problem with piecing dependencies together is that for that approach to work out, everything from the FreeCAD and its dependencies to the Python interpreter and its dependencies seems to need to be built on the same versions of the libraries that they depend on to avoid causing segmentation faults that brings everything to a crashing halt. This means that the idea of grabbing FreeCAD modules and libraries it depends on from a Flatpak installation is in theory not horrible, as all parts are built together using the same library versions. I just couldn't make it work out, presumably due to how the included libraries are located and difficulty identifying an executable for the included Python interpreter. In the end, I looked into the contents of the FreeCAD AppImage, and that turned out to have everything needed in a folder structure that appears to be very friendly to what PyCharm and Python expects from modules and libraries.
This is what I did to get FreeCAD to work with PyCharm and virtualenv:
Download FreeCAD AppImage
https://www.freecadweb.org/downloads.php
Make AppImage executable
chmod -v +x ~/Downloads/FreeCAD_*.AppImage
Create folder for extracting AppImage
mkdir -v ~/Documents/freecad_appimage
Extract AppImage from folder (note: this expands to close to 30000 files requiring in excess of 2 GB disk space)
cd ~/Documents/freecad_appimage
~/Downloads/./FreeCAD_*.AppImage --appimage-extract
Create folder for PyCharm project
mkdir -v ~/Documents/pycharm_freecad_project
Create pycharm project using Python interpreter from extracted AppImage
Location: ~/Documents/pycharm_freecad_project
New environment using: Virtualenv
Location: ~/Documents/pycharm_freecad_project/venv
Base interpreter: ~/Documents/freecad_appimage/squashfs-root/usr/bin/python
Inherit global site-packages: False
Make available to all projects: False
Add folder containing FreeCAD.so library as Content Root to PyCharm Project Structure and mark as Sources (by doing so, you shouldn't have to set PYTHONPATH or sys.path values, as PyCharm provides module location information to the interpreter)
File: Settings: Project: Project Structure: Add Content Root
~/Documents/freecad_appimage/squashfs-root/usr/lib
After this PyCharm is busy indexing files for a while.
Open Python Console in PyCharm and run command to check basic functioning
import FreeCAD
Create python script with example functionality
import FreeCAD
vec = FreeCAD.Base.Vector(0, 0, 0)
print(vec)
Run script
Debug script
All FreeCAD functionality I have used in my scripts so far has worked. However, one kink seems to be that the FreeCAD module needs to be imported before the Path module. Otherwise the Python interpreter exits with code 139 (interrupted by signal 11: SIGSEGV).
There are a couple of issues with PyCharm: It is showing a red squiggly line under the import name and claiming that an error has happened because "No module named 'FreeCAD'", even though the script is running perfectly. Also, PyCharm fails to provide code completion in the code view, even though it manages to do so in it's Python Console. I am creating new questions to address those issues and will update info here if I find a solution.
I set up a virtual environment in python 3.7.2 using "python -m venv foldername". I installed PIL in that folder. Importing PIL works from the terminal, but when I try to import it in VS code, I get an ImportError. Does anyone know how to get VS code to recognize that module?
I've tried switching interpreters, but the problem persists.
I ended up changing the python.venvpath setting to a different folder, and then moving the virtual env folder(The one with my project in it) to that folder. After restarting VS code, it worked.
I created a venv using python3.6 on my mac os in this folder
/Users/kim/Documents/Apps/PythonApps/python36-miros-a3
I ran a pip install pylint after I activated the virtual env
My workspace is in /Users/kim/Documents/Apps/WebApps/miros-a3
Inside my vscode workspace, I have the following Workspace settings
{
"folders": [
{
"path": "."
}
],
"settings": {
"python.pythonPath": "/Users/kim/Documents/Apps/PythonApps/python36-miros-a3/bin/python3.6",
"python.venvPath": "/Users/kim/Documents/Apps/PythonApps"
}
}
I have tried setting a custom path for the pylint and also changing the venvpath.
The pylint kept complaining about the import statement saying it does not exist.
As you can see, they are in the same folder, and I can definitely execute my python files.
What can I do to avoid these kind of false positive import errors?
I have also tried the following:
go to commandline turn on the virtual env and then type code to activate the vscode as recommended here https://code.visualstudio.com/docs/setup/mac
also tried this https://donjayamanne.github.io/pythonVSCodeDocs/docs/troubleshooting_linting/
Pylint has some quirks. In this case it doesn't know where to find your module because it's in subdirectory of your venv path. To solve this:
Put this setting in your workspace or folder settings:
"python.linting.pylintArgs": [
"--init-hook",
"import sys; sys.path.append('<path to folder your module is in>')"
]
or, maybe better
Generate .pylintrc file. From integrated terminal with venv activated run:
pylint --generate-rcfile > .pylintrc
then open the generated file and uncomment the init-hook= part to be:
init-hook='import sys; sys.path.append("<path to folder you module is in>")'
Read the .pylintrc and tweak settings if you wish. In both cases path should point to your 'database' folder.
After learning about pylint settings, do it the right way:
from database.database_dispatcher import ...
See this answer by Anthony Sottile.
To me, pylint is correct in flagging this error here
the top level module is database (it contains an __init__.py file)
Your import should look like (fully absolute)
from database.database_dispatcher import ...
or (explicit relative) (yes! the . before the module name is intentional)
from .database_dispatcher import ...
My follow-up guess is that you're currently invoking your script as python ./database/main.py ... which is putting ./database at the beginning of sys.path so it would appear that your imports are working correctly -- this is side-stepping your module structure however. You should be invoking your script using python -m database.main ... instead.
Note that implicit relative imports were removed in python 3.x -- though this (imo) wart of script sys.path insertion remains.
Just my $0.02 on how I fixed it in my situation.
My problem was totally related to having pylint installed globally, and coding in a venv. vscode was trying to use the globally installed pylint which simply was not aware of dependencies I installed in my Python venv. This answer solved my problem. It points here which explained how to configure vscode to run using the venv for my project. Once i did that vscode immediately threw a warning saying I had no linting tool installed and prompted me to install one. Once that was done my linting false-positives went away.
Expanding on Gatmando's answer, you need to tell vscode to use the pylint in your .env instead of the global pylint:
In your workspace settings file: .vscode/settings.json, add python.linting.pylintPath and point it to pylint package in your virtualenv:
{
"python.pythonPath": ".env/bin/python",
"python.linting.pylintPath": ".env/bin/pylint"
}
Another option is to create a .env file in the vscode project root dir:
.env file:
PYTHONPATH=.
Some caveats of using the .env file:
It will be used for anything that the vscode ide runs (e.g. pylint, pytest), so the PYTHONPATH may clash between different tools.
You could specify multiple dirs to PYTHONPATH to accommodate multiple tools by using separators. (Example: PYTHONPATH=src:. ) However, this is not cross-OS because the Windows separator is ; while Linux/Mac is :
If the .env PYTHONPATH solves your issue, then great. But due to the caveats, I ended up using the pylint --init-hook method because it only affects pylint, and it can be cross-OS:
.vscode/settings.json
"python.linting.pylintArgs": [
"--init-hook",
"import sys, os; sys.path.insert(0, os.path.join('src', 'foo'))"
]
I have a python application that I built on Kivy for the GUI, and I separated that file from the RNN model. I just import that file directly with:
from keras_network import Network
I tried using the official trick to deploying using pyinstaller, but that doesn't work, as my application crashes, when running the final .exe file generated. I even made appropriate changes to the .spec file.
Isn't there an easier way to package this application in Python 3.6? (like SqueezeTool, which is too old, and isn't working in Python 3.6)
Can I compile the python files to .pyc files and have all dependencies statically linked? And why is this still such a big pain, when Python is so popular?
Ok... I finally got it to work.
I came up with this solution after I implemented the whole project in PyQt5, and while trying to package it with PyInstaller.
Here is how I got it to work (for people who could have any problems later):
Used python 3.5
Install Windows 10 SDK for some missing files that should've been with MSVC 2015.
Install the dev version PyInstaller from:
pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip --upgrade
Used the command to include my data files, as well as including hidden import h5py:
pyinstaller --add-data keras_model.h5;. --add-data TrainingData.txt;. --hidden-import=h5py --hidden-import=h5py.defs --hidden-import=h5py.utils --hidden-import=h5py.h5ac --hidden-import=h5py._proxy project-name.py
Then edited the .spec file generated. Added "from kivy.deps import sdl2, glew" without the quotes in the beginning of the file after the comment. Also added "*[Tree(p) for p in (sdl2.dep_bins + glew.dep_bins)]," without the quotes in the COLLECT() function call as the 6th argument.
Then used the following command to rebuild using .spec file:
pyinstaller --add-data keras_model.h5;. --add-data TrainingData.txt;. --hidden-import=h5py --hidden-import=h5py.defs --hidden-import=h5py.utils --hidden-import=h5py.h5ac --hidden-import=h5py._proxy project-name.spec
Then the built application's executable was present in the 'dist\project-name\' directory. Double clicking the application lead to the application executing perfectly.
Hope this helps anyone stuck in the same place.
I got the next problem.
During the last 3 months, I was building an application in Python 3.5 to test luminaries.
For this application I used the next libraries, "tkinter", "sys", "threading", "os", "time", "shutil" that are included on the main Python 3.5 interpreter. In addition to this system libraries, I use "pyserial", "PyGreSQL", "PIL", "win32com", and "qrcode".
The program will be used on many PCs, so I need to compile all the code to make a fully functional application in every PC that is installed.
I tried using PyInstaller, but it not works for me, for a reason that I don't understand, I can not make it work.
So, due that I think there is no more way to "compile" my python 3.5 app to an exe (all the solutions only works in Python 2.x), I think the solution is only two ways to resolve.
The first solution is, to copy the Python35 folder, that got all the libraries that I have already installed into another PC(this is because every single that I installed give me problems at the installation, specially pygresql) and if I can make an "Installer" that overwrite this Python35 folder into the new PC and create a link icon to the desktop that runs the .py script. (Simulate an APP is installed).
Or the second solution, rewrite all code into 2.x syntax and try installing again libraries for python 2.x, and make all the code functional again...
Which one do you recommend me... Or if you know any way that I could do this... I would be very grateful if you help me.
Thanks!
I've used CX_Freeze on my programs in the past, so far I have only used it with python 3.4.3, but I see no reason why it shouldn't work with python 3.5.
You can also install it with pip:
pip install CX_Freeze
See here for a tutorial.
I'm out at the moment, so I'll comment the code I use when I next have my computer.
I hope this helps.
EDIT:
Here is a version of the code (for batch):
set PATH=%PATH%;C:\Python34\
python "setup.py" build
pause
to use this code, you will need to make a setup script pointing to your program, an example is below:
import sys
from cx_Freeze import setup, Executable
# Dependencies are automatically detected, but it might need fine tuning. This example will include the os package, and omit the tkinter package
build_exe_options = {"packages": ["os"], "excludes": ["tkinter"]}
# GUI applications require a different base on Windows (the default is for a
# console application).
base = None
if sys.platform == "win32":
base = "Win32GUI"
setup( name = "test",
version = "0.1",
description = "My GUI application!",
options = {"build_exe": build_exe_options},
executables = [Executable("test.py", base=base)])
EDIT2:
Although the above code does work, the code I use is:
cxfreeze "%cd%\File.py" --target-dir "%cd%\File\"