I used a virtualbox with ubuntu-16, to install caffe in python 2.6. As I wanted to use py2exe, I needed to change python version to 3.6. When this was done, caffe import code stopped working. Here is the error message:
ImportError: libcaffe.so.1.0.0-rc5: cannot open shared object file: No such file
or directory
Do I have to rebuild caffe? Or is something else that need to be changed?
Here is the full image of the error:
Yes, you need to rebuild Caffe. The Python 2 and Python 3 libraries are not compatible. Not all of the file names are the same (different / added software organization). This rebuild requires that you place a single version of Python first in all search paths: make, compile, load, ...
When I need to do this, I check with my SysOps: there's always something I forget.
Related
How do I setup a Python virtual environment with the FreeCAD library embedded as to enable import as a module into scripts?
I would like to avoid using the FreeCAD GUI as well as being dependent on having FreeCAD installed on the system, when working with Python scripts that use FreeCAD libraries to create and modify 3D geometry. I hope a Python virtual environment can make that possible.
I am working in PyCharm 2021.1.1 with Python 3.8 in virtualenv on Debian 10.
I started out with FreeCAD documentation for embedding in scripts as a basis:
https://wiki.freecadweb.org/Embedding_FreeCAD
In one attempt, I downloaded and unpacked .deb packages from Debian, taking care to get the correct versions required by each dependency. In another attempt, I copied the contents of a FreeCAD flatpak install, as it should contain all the libraries that FreeCAD depends on.
After placing the libraries to be imported in the virtual maching folder, I have pointed to them with sys.path.append() as well as PyCharm's Project Structure tool in various attempts. In both cases the virtual environment detects where FreeCAD.so is located, but fails to find any of its dependencies, even when located in the same folder. When importing these dependencies explicitly, each of them have the same issue. This leads to a dead end when an import fails because it does not define a module export function according to Python:
ImportError: dynamic module does not define module export function (PyInit_libnghttp2)
I seem to be looking at a very long chain of broken dependencies, even though I make the required libraries available and inform Python where they are located.
I would appreciate either straight up instructions for how to do this or pointers to documentation that describes importing FreeCAD libraries in Python virtual environments, as I have not come across anything that specific yet.
I came across a few prior questions which seemed to have similar intent, but no answers:
Embedding FreeCAD in python script
Is it possible to embed Blender/Freecad in a python program?
Similar questions for Conda focus on importing libraries from the host system rather than embedding them in the virtual environment:
Incude FreeCAD in system path for just one conda virtual environment
Other people's questions on FreeCAD forums went unanswered:
https://forum.freecadweb.org/viewtopic.php?t=27929
EDIT:
Figuring this out was a great learning experience. The problem with piecing dependencies together is that for that approach to work out, everything from the FreeCAD and its dependencies to the Python interpreter and its dependencies seems to need to be built on the same versions of the libraries that they depend on to avoid causing segmentation faults that brings everything to a crashing halt. This means that the idea of grabbing FreeCAD modules and libraries it depends on from a Flatpak installation is in theory not horrible, as all parts are built together using the same library versions. I just couldn't make it work out, presumably due to how the included libraries are located and difficulty identifying an executable for the included Python interpreter. In the end, I looked into the contents of the FreeCAD AppImage, and that turned out to have everything needed in a folder structure that appears to be very friendly to what PyCharm and Python expects from modules and libraries.
This is what I did to get FreeCAD to work with PyCharm and virtualenv:
Download FreeCAD AppImage
https://www.freecadweb.org/downloads.php
Make AppImage executable
chmod -v +x ~/Downloads/FreeCAD_*.AppImage
Create folder for extracting AppImage
mkdir -v ~/Documents/freecad_appimage
Extract AppImage from folder (note: this expands to close to 30000 files requiring in excess of 2 GB disk space)
cd ~/Documents/freecad_appimage
~/Downloads/./FreeCAD_*.AppImage --appimage-extract
Create folder for PyCharm project
mkdir -v ~/Documents/pycharm_freecad_project
Create pycharm project using Python interpreter from extracted AppImage
Location: ~/Documents/pycharm_freecad_project
New environment using: Virtualenv
Location: ~/Documents/pycharm_freecad_project/venv
Base interpreter: ~/Documents/freecad_appimage/squashfs-root/usr/bin/python
Inherit global site-packages: False
Make available to all projects: False
Add folder containing FreeCAD.so library as Content Root to PyCharm Project Structure and mark as Sources (by doing so, you shouldn't have to set PYTHONPATH or sys.path values, as PyCharm provides module location information to the interpreter)
File: Settings: Project: Project Structure: Add Content Root
~/Documents/freecad_appimage/squashfs-root/usr/lib
After this PyCharm is busy indexing files for a while.
Open Python Console in PyCharm and run command to check basic functioning
import FreeCAD
Create python script with example functionality
import FreeCAD
vec = FreeCAD.Base.Vector(0, 0, 0)
print(vec)
Run script
Debug script
All FreeCAD functionality I have used in my scripts so far has worked. However, one kink seems to be that the FreeCAD module needs to be imported before the Path module. Otherwise the Python interpreter exits with code 139 (interrupted by signal 11: SIGSEGV).
There are a couple of issues with PyCharm: It is showing a red squiggly line under the import name and claiming that an error has happened because "No module named 'FreeCAD'", even though the script is running perfectly. Also, PyCharm fails to provide code completion in the code view, even though it manages to do so in it's Python Console. I am creating new questions to address those issues and will update info here if I find a solution.
I am trying to install a package I have written by running
python setup.py install
whilst in the package directory. My folder structure is as follows:
package
-module
--__init__.py
--parent_class.py
--child_class.py
-setup.py
Here '-' denotes the folder level with package being the root directory.
When running the installation works as expected until:
Installed c:\users\me\appdata\local\continuum\anaconda3\lib\site-
packages\package-0.1-py3.7.egg
Processing dependencies for package==0.1
Searching for json
Reading https://pypi.org/simple/json/
Couldn't find index page for 'json' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.org/simple/
No local packages or working download links found for json
error: Could not find suitable distribution for Requirement.parse('json')
I don't believe this is an issue with the json package installation as:
I know I have it installed (I can run 'import json' in my env)
and because if I remove it.
If i remove 'json' from install_requires=['pandas', 'numpy', 'datetime', 'os', 'copy', 'json'] it will simply return the same error, this time with the 'copy' and then the 'os' package.
Thanks in advance
I believe json is a built-in module in Python 2 and 3 (e.g., like open). Thus, it's usually included in all Python distributions (that is, anyone who has Python probably also has the json module).
Thus, you should be able to omit json from your requirements (while still importing it in your code). Since json is a built-in module, you can assume that anyone who has Python installed will also have json.
I found this question that is also about installing the json module. The second answer notes that minimal Python installs (e.g., python-minimal in Ubuntu) may not include json, however. You should note that the linked question concerns Python 2 and not 3.
Therefore, if a user lacks json on their system, they should be able to install it through their system's package manager (in Linux), and not pip. In Windows, I would assume that all Python installations will include built-in modules (though I don't know for certain). Regardless, if a user doesn't have json installed, then I think it's up to the user to install it.
The same applies to os, copy, and datetime. Only pandas and numpy need to be included in your requirements.
You can find a list of built-in modules (for Python 3) here. Again, the modules listed here are ones that are not hosted on pypi (thus not available through pip install ..., conda install ..., etc.), but ones that are usually included in all Python distributions.
As an aside: this is my first answer here. Please let me know if I can improve it.
I'm working on a pure x64 version of my software. Our installer for the x86 version is NSIS and my software is built using VS2012. I found online that in order to build x64 installers using NSIS you have to build NSIS (and all plugins/etc) from source. That is the end goal. However, right now I'm having issues just building x86 (before moving on to x64) using the instructions in the NSIS docs. https://nsis.sourceforge.io/Docs/AppendixG.html#build_windows
I'm trying to build NSIS v3.0.4 which uses SCons to build. I've installed scons using pip (SCons version 3.1.1).
However, when trying to build NSIS, I get the following (using python 3.8.0).
C:\Source\nsis\nsis-code-r7069-NSIS-tags-v304>scons ZLIB_W32=C:\Source\zlib-1.2.7 MSTOOLKIT=yes
scons: Reading SConscript files ...
TypeError: cannot use a string pattern on a bytes-like object:
File "C:\Source\nsis\nsis-code-r7069-NSIS-tags-v304\SConstruct", line 263:
for v in re.compile(r'^\\H\{[v]?(\S+)\}', re.M).finditer(File('#/Docs/src/history.but').get_contents()): # Try to parse the Halibut history file
the section of the SConstruct file is as follows (starting at line 260 so the for loop starts at 263):
if not defenv.has_key('VER_PACKED'):
import re
found = None
for v in re.compile(r'^\\H\{[v]?(\S+)\}', re.M).finditer(File('#/Docs/src/history.but').get_contents()): # Try to parse the Halibut history file
if v and not found:
v = v.group(1).split('.')
if len(v) >= 2:
mi = int(re.search(r'\d+', v[1]).group())
if mi < 1: mi = 1 # Make sure we can subtract 1 from the minor number so trunk stays below the next release
defenv['VER_PACKED'] = '0x%0.2i%0.3i%0.2i%0.1i' % (int(re.search(r'\d+', v[0]).group()), mi - 1, 66, 6)
if int(defenv['VER_PACKED'], 0) >= int('0x03000000', 0):
found = v
if not found:
defenv['VER_PACKED'] = '0x%0.2i%0.3i%0.2i%0.1i' % (3, 3, 42, 0) # Default to a version number we never used
print('WARNING: VER_PACKED not set, defaulting to %s!' % defenv['VER_PACKED'])
At this point, I'm not sure if there is a dependency I'm missing or if this is an issue with the build script and needs to be reported at the NSIS issue tracker. Ideas?
-UPDATE 1-
Based on suggestions from #Anders, first I tried to build the head of the repo (r7132 trunk) instead of v3.0.4 using Python3. This got me farther but still failed on what appears to be Python2 vs 3 syntax errors in mstoolkit.py (might be a symptom of another issue rather than the cause though). I didn't investigate this too much further yet.
So next, I went back to Python 2.7. This got me further but the mstoolkit.py appears to be dependent on VS2003 implementation. I duplicated the VCToolkitInstallDir environment variable on my machine using the contents of the VS110COMNTOOLS (C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\Tools) but that fails also:
File "C:\Source\IQClient_Interim\Shared\third_party\nsis\nsis-code-r7132-NSIS-tags-v304\SCons\Tools\mstoolkit.py", line 87, in get_msvctoolkit_paths
raise SCons.Errors.InternalError, "The Platform SDK directory was not found in the registry or in the `MSSdk` environment variable."
InternalError: The Platform SDK directory was not found in the registry or in the `MSSdk` environment variable.
I don't know what the VS2003 VCToolkitInstallDir ENVVAR was pointing to to fix this.
With all that said, I just noticed that Anders said that I might also have to use an older version of scons as well with python2.7 so I suppose I'll spend some time investigating that (along with looking into #bdbaddog's solution).
-UPDATE 2-
it was the mstoolkit directive (see #Anders answer's comments)
I got it working after that while using py 2.7 for NSIS 3.0.4.
Now on to x64 build but that's a different story so I'll post a different question.
NSIS v3.04 does not support Python 3, you have to grab the latest source code from SVN. v3.05 is coming out soon with support for Python 3.
VS2012 is old enough that you should be able to use Python 2.7 and a older version of SCons as an alternative.
As a final alternative, you can try to apply this diff to v3.04.
The issue you're running into is twofold.
Python > 3.0 strings are no longer bytes. They are unicode
SCons which supports Python > 3.0 (SCons >3.0 supports Python 2.7 and Python > 3.5.0)
In order to support this Node()'s (SCons's way of representing files, directories, and values which feed into a build) now have two methods get_contents() and get_text_contents().
Since you're using Python 3.8.0, get_contents() will return bytes, which explains the error you've received.
The proper way to resolve it is to change this line:
for v in re.compile(r'^\\H\{[v]?(\S+)\}', re.M).finditer(File('#/Docs/src/history.but').get_contents()): # Try to parse the Halibut history file
to:
for v in re.compile(r'^\\H\{[v]?(\S+)\}', re.M).finditer(File('#/Docs/src/history.but').get_text_contents()): # Try to parse the Halibut history file
Please let me know if this doesn't resolve the issue.
Also, if you're willing to install Python 2.7 and reinstall SCons via
py -2.7 -mpip install scons
(Assuming you've install the py launcher when installing Python. Not sure if that's default with Python 3.8 and/or Python 2.7. If not then /python -m pip install scons. You might have to install pip as well. Once again not sure on the default for that.)
Then you can avoid changing the SConstruct file.
Hope this helps!
I have a python application that I built on Kivy for the GUI, and I separated that file from the RNN model. I just import that file directly with:
from keras_network import Network
I tried using the official trick to deploying using pyinstaller, but that doesn't work, as my application crashes, when running the final .exe file generated. I even made appropriate changes to the .spec file.
Isn't there an easier way to package this application in Python 3.6? (like SqueezeTool, which is too old, and isn't working in Python 3.6)
Can I compile the python files to .pyc files and have all dependencies statically linked? And why is this still such a big pain, when Python is so popular?
Ok... I finally got it to work.
I came up with this solution after I implemented the whole project in PyQt5, and while trying to package it with PyInstaller.
Here is how I got it to work (for people who could have any problems later):
Used python 3.5
Install Windows 10 SDK for some missing files that should've been with MSVC 2015.
Install the dev version PyInstaller from:
pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip --upgrade
Used the command to include my data files, as well as including hidden import h5py:
pyinstaller --add-data keras_model.h5;. --add-data TrainingData.txt;. --hidden-import=h5py --hidden-import=h5py.defs --hidden-import=h5py.utils --hidden-import=h5py.h5ac --hidden-import=h5py._proxy project-name.py
Then edited the .spec file generated. Added "from kivy.deps import sdl2, glew" without the quotes in the beginning of the file after the comment. Also added "*[Tree(p) for p in (sdl2.dep_bins + glew.dep_bins)]," without the quotes in the COLLECT() function call as the 6th argument.
Then used the following command to rebuild using .spec file:
pyinstaller --add-data keras_model.h5;. --add-data TrainingData.txt;. --hidden-import=h5py --hidden-import=h5py.defs --hidden-import=h5py.utils --hidden-import=h5py.h5ac --hidden-import=h5py._proxy project-name.spec
Then the built application's executable was present in the 'dist\project-name\' directory. Double clicking the application lead to the application executing perfectly.
Hope this helps anyone stuck in the same place.
I wanted to use the code:
import geodict_lib
locations = geodict_lib.find_locations_in_text(text)
But there seems to be no installer for geodict_lib. How do I install this is Anaconda 3.0 Python 3?
I know this is a year on, but perhaps I could help others who stumble on this. You'll need to place the files in the directory for modules that your installation of Python is monitoring.
First, download the .zip file from GitHub here.
Once you've done that, you can run the following at the command line or terminal:
conda list
This will provide the path to all installed packages in your installation of Python. Move the geodict.zip file you downloaded to that location. You might want to run which python as well (see here) since you may have a few different installations to check for.
Now when you run python import geodict in Python it should run without trouble!