How do I setup a Python virtual environment with the FreeCAD library embedded as to enable import as a module into scripts?
I would like to avoid using the FreeCAD GUI as well as being dependent on having FreeCAD installed on the system, when working with Python scripts that use FreeCAD libraries to create and modify 3D geometry. I hope a Python virtual environment can make that possible.
I am working in PyCharm 2021.1.1 with Python 3.8 in virtualenv on Debian 10.
I started out with FreeCAD documentation for embedding in scripts as a basis:
https://wiki.freecadweb.org/Embedding_FreeCAD
In one attempt, I downloaded and unpacked .deb packages from Debian, taking care to get the correct versions required by each dependency. In another attempt, I copied the contents of a FreeCAD flatpak install, as it should contain all the libraries that FreeCAD depends on.
After placing the libraries to be imported in the virtual maching folder, I have pointed to them with sys.path.append() as well as PyCharm's Project Structure tool in various attempts. In both cases the virtual environment detects where FreeCAD.so is located, but fails to find any of its dependencies, even when located in the same folder. When importing these dependencies explicitly, each of them have the same issue. This leads to a dead end when an import fails because it does not define a module export function according to Python:
ImportError: dynamic module does not define module export function (PyInit_libnghttp2)
I seem to be looking at a very long chain of broken dependencies, even though I make the required libraries available and inform Python where they are located.
I would appreciate either straight up instructions for how to do this or pointers to documentation that describes importing FreeCAD libraries in Python virtual environments, as I have not come across anything that specific yet.
I came across a few prior questions which seemed to have similar intent, but no answers:
Embedding FreeCAD in python script
Is it possible to embed Blender/Freecad in a python program?
Similar questions for Conda focus on importing libraries from the host system rather than embedding them in the virtual environment:
Incude FreeCAD in system path for just one conda virtual environment
Other people's questions on FreeCAD forums went unanswered:
https://forum.freecadweb.org/viewtopic.php?t=27929
EDIT:
Figuring this out was a great learning experience. The problem with piecing dependencies together is that for that approach to work out, everything from the FreeCAD and its dependencies to the Python interpreter and its dependencies seems to need to be built on the same versions of the libraries that they depend on to avoid causing segmentation faults that brings everything to a crashing halt. This means that the idea of grabbing FreeCAD modules and libraries it depends on from a Flatpak installation is in theory not horrible, as all parts are built together using the same library versions. I just couldn't make it work out, presumably due to how the included libraries are located and difficulty identifying an executable for the included Python interpreter. In the end, I looked into the contents of the FreeCAD AppImage, and that turned out to have everything needed in a folder structure that appears to be very friendly to what PyCharm and Python expects from modules and libraries.
This is what I did to get FreeCAD to work with PyCharm and virtualenv:
Download FreeCAD AppImage
https://www.freecadweb.org/downloads.php
Make AppImage executable
chmod -v +x ~/Downloads/FreeCAD_*.AppImage
Create folder for extracting AppImage
mkdir -v ~/Documents/freecad_appimage
Extract AppImage from folder (note: this expands to close to 30000 files requiring in excess of 2 GB disk space)
cd ~/Documents/freecad_appimage
~/Downloads/./FreeCAD_*.AppImage --appimage-extract
Create folder for PyCharm project
mkdir -v ~/Documents/pycharm_freecad_project
Create pycharm project using Python interpreter from extracted AppImage
Location: ~/Documents/pycharm_freecad_project
New environment using: Virtualenv
Location: ~/Documents/pycharm_freecad_project/venv
Base interpreter: ~/Documents/freecad_appimage/squashfs-root/usr/bin/python
Inherit global site-packages: False
Make available to all projects: False
Add folder containing FreeCAD.so library as Content Root to PyCharm Project Structure and mark as Sources (by doing so, you shouldn't have to set PYTHONPATH or sys.path values, as PyCharm provides module location information to the interpreter)
File: Settings: Project: Project Structure: Add Content Root
~/Documents/freecad_appimage/squashfs-root/usr/lib
After this PyCharm is busy indexing files for a while.
Open Python Console in PyCharm and run command to check basic functioning
import FreeCAD
Create python script with example functionality
import FreeCAD
vec = FreeCAD.Base.Vector(0, 0, 0)
print(vec)
Run script
Debug script
All FreeCAD functionality I have used in my scripts so far has worked. However, one kink seems to be that the FreeCAD module needs to be imported before the Path module. Otherwise the Python interpreter exits with code 139 (interrupted by signal 11: SIGSEGV).
There are a couple of issues with PyCharm: It is showing a red squiggly line under the import name and claiming that an error has happened because "No module named 'FreeCAD'", even though the script is running perfectly. Also, PyCharm fails to provide code completion in the code view, even though it manages to do so in it's Python Console. I am creating new questions to address those issues and will update info here if I find a solution.
There is some code I want to turn into an exe. I have used Pyinstaller and I manage getting an exe that runs. The platform is Win10, Python version 3.6, devs with Spyder.
At some point in the code, the following lines pose an issue with the exe (not in script mode):
from formulas import parser as formulasparser
# code including classes
# in a function:
t = 'x<1' # any string that is a 'formula', here provided for the example
parser_ = formulasparser.Parser()
# the exe works so far, I have traced the code and can read the messages in the windows cmd
func = parser_.ast(t)
# nothing happens and no way to get a trace; there is an issue, and I have no clue which one as there
# is no error message.
So, it seems that the ast function won't work for some reason. Can anyone help me? I need to distribute this application and it does not work because of this right now.
Thanks for your help.
Well, someone seems to have faced a somewhat similar issue on the Pyinstaller mailing list.
So: a hook file is actually needed. I created a directory called pyinstallerhooks with a file called hook-formulas.py inside.
The file itself contains:
from PyInstaller.utils.hooks import collect_all
datas, binaries, hiddenimports = collect_all('formulas')
and this is it.
When calling pyinstaller, use the following command:
pyinstaller --additional-hooks-dir pathtoyourdir -F yoursrcipt.py
Pyinstaller fetches the relevant dependencies for the formulas package and it shoudl work.
System: Python 3.7 64x running on Windows 10. I also have Anaconda installed on my machine and I program through VS Code. All modules have been installed, verified installed, and are up-to-date.
I have written a script and compiled it into an .exe using pyinstaller. pyinstaller -wFi pyico.ico endpointapp3.py.The .exe works and runs on my machine but it's not meant for my computer. Previous iterations of this script have been compiled, tested, and worked appropriately on its destination computer, a Windows 7 machine with no python installed.
My latest version now includes pyserial, and works, again, on my machine, but I after compiling and creating an .exe file for the destination machine I encounter this error.
File "endpointapp2.py", line 9 in <module>
ModuleNotFoundError: No module named 'serial'
[3512] Failed to execute script endpointapp2.py
Line 9 is my import serial line
As I mentioned before, this executable runs on my machine, but not on the destination machine it was designed for. I don't want to install python, or python modules on the destination machine because I want the .exe to be all inclusive.
Questions:
Why am I getting this error? What does it mean?
Is there a work around that allows me to compile everything together (using pyinstaller) that will hold all the necessary module
information within the .exe?
Is there something else I should be adding to my original script that will allow me to work through this?
NOT A DUPLICATE - This post is for compiling and running python .exe's on another machine, independent of python, with no modules or python installed.
With some assistance and helpful advice (thanks martineau) I was able to solve my problem. When compiling with pyinstaller, it makes a .spec file that it actually uses to compile to .exe.
Here I found the necessary steps to make sure I was adding the files I needed for pyinstaller.
Here I found out how to properly add the module information into my compiling code.
Once pyinstaller had the correct info in the .spec file I was able to compile and execute on the destination machine as intended.
I solve this issue when i move my code into serial module location.
First find where serial module locate :
>>> import serial
>>> serial.__file__
'C:\\Program Files\\Python310\\lib\\site-packages\\serial\\__init__.py'
mine is above.
Then move your file into :
'C:\\Program Files\\Python310\\lib\\site-packages\\{your_file.py}'
Then run pyinstaller :
pyinstaller --onefile your_file.py --additional-hooks-dir serial\\__init__.py
Run your .exe from dist folder. No longer showing error.
The module is called pyserial, but the library call in the code is import serial. Pyinstaller is unable to complete the build because there truly is no module named serial.
To correct this you need to tell pyinstaller to import the correct package by including it in the .spec file under hiddenimports.
hiddenimports=['pyserial']
This will be true for any case where the package name and the import statement call are not the same.
I have a python application that I built on Kivy for the GUI, and I separated that file from the RNN model. I just import that file directly with:
from keras_network import Network
I tried using the official trick to deploying using pyinstaller, but that doesn't work, as my application crashes, when running the final .exe file generated. I even made appropriate changes to the .spec file.
Isn't there an easier way to package this application in Python 3.6? (like SqueezeTool, which is too old, and isn't working in Python 3.6)
Can I compile the python files to .pyc files and have all dependencies statically linked? And why is this still such a big pain, when Python is so popular?
Ok... I finally got it to work.
I came up with this solution after I implemented the whole project in PyQt5, and while trying to package it with PyInstaller.
Here is how I got it to work (for people who could have any problems later):
Used python 3.5
Install Windows 10 SDK for some missing files that should've been with MSVC 2015.
Install the dev version PyInstaller from:
pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip --upgrade
Used the command to include my data files, as well as including hidden import h5py:
pyinstaller --add-data keras_model.h5;. --add-data TrainingData.txt;. --hidden-import=h5py --hidden-import=h5py.defs --hidden-import=h5py.utils --hidden-import=h5py.h5ac --hidden-import=h5py._proxy project-name.py
Then edited the .spec file generated. Added "from kivy.deps import sdl2, glew" without the quotes in the beginning of the file after the comment. Also added "*[Tree(p) for p in (sdl2.dep_bins + glew.dep_bins)]," without the quotes in the COLLECT() function call as the 6th argument.
Then used the following command to rebuild using .spec file:
pyinstaller --add-data keras_model.h5;. --add-data TrainingData.txt;. --hidden-import=h5py --hidden-import=h5py.defs --hidden-import=h5py.utils --hidden-import=h5py.h5ac --hidden-import=h5py._proxy project-name.spec
Then the built application's executable was present in the 'dist\project-name\' directory. Double clicking the application lead to the application executing perfectly.
Hope this helps anyone stuck in the same place.
This question already has answers here:
Create a single executable from a Python project [closed]
(3 answers)
Closed 1 year ago.
I'm building a Python application and don't want to force my clients to install Python and modules.
So, is there a way to compile a Python script to be a standalone executable?
You can use PyInstaller to package Python programs as standalone executables. It works on Windows, Linux, and Mac.
PyInstaller Quickstart
Install PyInstaller from PyPI:
pip install pyinstaller
Go to your program’s directory and run:
pyinstaller yourprogram.py
This will generate the bundle in a subdirectory called dist.
pyinstaller -F yourprogram.py
Adding -F (or --onefile) parameter will pack everything into single "exe".
pyinstaller -F --paths=<your_path>\Lib\site-packages yourprogram.py
running into "ImportError" you might consider side-packages.
pip install pynput==1.6.8
still runing in Import-Erorr - try to downgrade pyinstaller - see Getting error when using pynput with pyinstaller
For a more detailed walkthrough, see the manual.
You can use py2exe as already answered and use Cython to convert your key .py files in .pyc, C compiled files, like .dll in Windows and .so on Linux.
It is much harder to revert than common .pyo and .pyc files (and also gain in performance!).
You might wish to investigate Nuitka. It takes Python source code and converts it in to C++ API calls. Then it compiles into an executable binary (ELF on Linux). It has been around for a few years now and supports a wide range of Python versions.
You will probably also get a performance improvement if you use it. It is recommended.
Yes, it is possible to compile Python scripts into standalone executables.
PyInstaller can be used to convert Python programs into stand-alone executables, under Windows, Linux, Mac OS X, FreeBSD, Solaris, and AIX. It is one of the recommended converters.
py2exe converts Python scripts into only executable on the Windows platform.
Cython is a static compiler for both the Python programming language and the extended Cython programming language.
I would like to compile some useful information about creating standalone files on Windows using Python 2.7.
I have used py2exe and it works, but I had some problems.
It has shown some problems for creating single files in Windows 64 bits: Using bundle_files = 1 with py2exe is not working;
It is necessary to create a setup.py file for it to work. http://www.py2exe.org/index.cgi/Tutorial#Step2;
I have had problems with dependencies that you have to solve by importing packages in the setup file;
I was not able to make it work together with PyQt.
This last reason made me try PyInstaller http://www.pyinstaller.org/.
In my opinion, it is much better because:
It is easier to use.
I suggest creating a .bat file with the following lines for example (pyinstaller.exe must be in in the Windows path):
pyinstaller.exe --onefile MyCode.py
You can create a single file, among other options (https://pyinstaller.readthedocs.io/en/stable/usage.html#options).
I had only one problem using PyInstaller and multiprocessing package that was solved by using this recipe: https://github.com/pyinstaller/pyinstaller/wiki/Recipe-Multiprocessing.
So, I think that, at least for python 2.7, a better and simpler option is PyInstaller.
And a third option is cx_Freeze, which is cross-platform.
pyinstaller yourfile.py -F --onefile
This creates a standalone EXE file on Windows.
Important note 1: The EXE file will be generated in a folder named 'dist'.
Important note 2: Do not forget --onefile flag
You can install PyInstaller using pip install PyInstaller
NOTE: In rare cases there are hidden dependencies...so if you run the EXE file and get missing library error (win32timezone in the example below) then use something like this:
pyinstaller --hiddenimport win32timezone -F "Backup Program.py"
I like PyInstaller - especially the "windowed" variant:
pyinstaller --onefile --windowed myscript.py
It will create one single *.exe file in a distination/folder.
You may like py2exe. You'll also find information in there for doing it on Linux.
Use py2exe.... use the below set up files:
from distutils.core import setup
import py2exe
from distutils.filelist import findall
import matplotlib
setup(
console = ['PlotMemInfo.py'],
options = {
'py2exe': {
'packages': ['matplotlib'],
'dll_excludes': ['libgdk-win32-2.0-0.dll',
'libgobject-2.0-0.dll',
'libgdk_pixbuf-2.0-0.dll']
}
},
data_files = matplotlib.get_py2exe_datafiles()
)
I also recommend PyInstaller for better backward compatibility such as Python 2.3 - 2.7.
For py2exe, you have to have Python 2.6.
For Python 3.2 scripts, the only choice is cx_Freeze. Build it from sources; otherwise it won't work.
For Python 2.x I suggest PyInstaller as it can package a Python program in a single executable, unlike cx_Freeze which outputs also libraries.
Since it seems to be missing from the current list of answers, I think it is worth mentioning that the standard library includes a zipapp module that can be used for this purpose. Its basic usage is just compressing a bunch of Python files into a zip file with extension .pyz than can be directly executed as python myapp.pyz, but you can also make a self-contained package from a requirements.txt file:
$ python -m pip install -r requirements.txt --target myapp
$ python -m zipapp -p "interpreter" myapp
Where interpreter can be something like /usr/bin/env python (see Specifying the Interpreter).
Usually, the generated .pyz / .pyzw file should be executable, in Unix because it gets marked as such and in Windows because Python installation usually registers those extensions. However, it is relatively easy to make a Windows executable that should work as long as the user has python3.dll in the path.
If you don't want to require the end user to install Python, you can distribute the application along with the embeddable Python package.
py2exe will make the EXE file you want, but you need to have the same version of MSVCR90.dll on the machine you're going to use your new EXE file.
See Tutorial for more information.
You can find the list of distribution utilities listed at Distribution Utilities.
I use bbfreeze and it has been working very well (yet to have Python 3 support though).
Not exactly a packaging of the Python code, but there is now also Grumpy from Google, which transpiles the code to Go.
It doesn't support the Python C API, so it may not work for all projects.
Using PyInstaller, I found a better method using shortcut to the .exe rather than making --onefile. Anyway, there are probably some data files around and if you're running a site-based app then your program depends on HTML, JavaScript, and CSS files too. There isn't any point in moving all these files somewhere... Instead what if we move the working path up?
Make a shortcut to the EXE file, move it at top and set the target and start-in paths as specified, to have relative paths going to dist\folder:
Target: %windir%\system32\cmd.exe /c start dist\web_wrapper\web_wrapper.exe
Start in: "%windir%\system32\cmd.exe /c start dist\web_wrapper\"
We can rename the shortcut to anything, so renaming to "GTFS-Manager".
Now when I double-click the shortcut, it's as if I python-ran the file! I found this approach better than the --onefile one as:
In onefile's case, there's a problem with a .dll missing for the Windows 7 OS which needs some prior installation, etc. Yawn. With the usual build with multiple files, no such issues.
All the files that my Python script uses (it's deploying a tornado web server and needs a whole freakin' website worth of files to be there!) don't need to be moved anywhere: I simply create the shortcut at top.
I can actually use this exact same folder on Ubuntu (run python3 myfile.py) and Windows (double-click the shortcut).
I don't need to bother with the overly complicated hacking of .spec file to include data files, etc.
Oh, remember to delete off the build folder after building. It will save on size.
Use Cython to convert to C, compile, and link with GCC.
Another could be, make the core functions in C (the ones you want to make hard to reverse), compile them and use Boost.Python to import the compiled code (plus you get a much faster code execution). Then use any tool mentioned to distribute.
I'm told that PyRun is also an option. It currently supports Linux, FreeBSD and Mac OS X.