I'm trying to compile the GitHub project: https://github.com/commaai/openpilot, I'm getting an error when creating the SCons environment (the call to Environment()), it points this line :
Environement(
# Other options ...
tools=["default", "cython", "compilation_db"
)
The result of the scons is then :
scons: Reading SConscript files ...
EnvironmentError: No module named compilation_db:
File "/home/skoumad/openpilot/master/SConstruct", line 213:
"compilation_db"
File "/usr/lib/scons/SCons/Environment.py", line 982:
apply_tools(self, tools, toolpath)
File "/usr/lib/scons/SCons/Environment.py", line 107:
env.Tool(tool)
File "/usr/lib/scons/SCons/Environment.py", line 1788:
tool = SCons.Tool.Tool(tool, toolpath, **kw)
File "/usr/lib/scons/SCons/Tool/__init__.py", line 118:
module = self._tool_module()
File "/usr/lib/scons/SCons/Tool/__init__.py", line 215:
raise SCons.Errors.EnvironmentError(error_string)
I tried to install compilation_db using : https://pypi.org/project/scons-compiledb/0.4.7/
but still the same error :/.
Any idea on how to install this missing module ??
Regards.
You likely have a version of SCons older than 4.0.0
The compilation_db tool was added in 4.0.0
See release notice
If you're distro(linux,python, macports, etc) doesn't have 4.0.0 or newer, then I'd advise setting up a python virtualenv (no this is not a VM, it's just a tool to create a clean python environment to install packages in)
Here's how to do that:
# assuming posix system, for win32, of course change the path to windows correct syntax
# also the python below should be 3.5 or newer, generally it's best to use the newest installed
python -m venv ~/sconsvenv
. ~/sconsvenv/bin/activate
pip install scons
scons --version
# should yield
SCons by Steven Knight et al.:
SCons: v4.1.0.post1.dc58c175da659d6c0bb3e049ba56fb42e77546cd, 2021-01-20 04:32:28, by bdbaddog on ProDog2020
SCons path: ['/Users/bdbaddog/sconsvenv/lib/python3.8/site-packages/SCons']
Copyright (c) 2001 - 2021 The SCons Foundation
Related
I'm using Python3 bindings for virtualbox (pyvbox). It works with the initial python3 version of my system which is 3.7. Now, I would like to use the very same bindings with python3.8. I have installed all needed python3.8 packages, but i get this error:
$ python3
Python 3.8.0 (default, Oct 28 2019, 16:14:01)
[GCC 9.2.1 20191008] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import virtualbox
>>> _vbox = virtualbox.VirtualBox()
m=VBoxPython3_8 x=No module named 'VBoxPython3_8'
m=VBoxPython3 x=No module named 'VBoxPython3'
m=VBoxPython x=/usr/lib/virtualbox/VBoxPython.so: undefined symbol: _Py_ZeroStruct
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home//.local/lib/python3.8/site-packages/virtualbox/library_ext/vbox.py", line 22, in __init__
manager = virtualbox.Manager()
File "/home//.local/lib/python3.8/site-packages/virtualbox/__init__.py", line 145, in __init__
self.manager = vboxapi.VirtualBoxManager(mtype, mparams)
File "/usr/local/lib/python3.8/dist-packages/vboxapi/__init__.py", line 989, in __init__
self.platform = PlatformXPCOM(dPlatformParams)
File "/usr/local/lib/python3.8/dist-packages/vboxapi/__init__.py", line 750, in __init__
import xpcom.vboxxpcom
File "/usr/lib/virtualbox/sdk/bindings/xpcom/python/xpcom/vboxxpcom.py", line 78, in <module>
raise Exception('Cannot find VBoxPython module (tried: %s)' % (', '.join(_asVBoxPythons),))
Exception: Cannot find VBoxPython module (tried: VBoxPython3_8, VBoxPython3, VBoxPython)
I have found that these modules have to be in /usr/lib/virtualbox, which currently contains only these files:
$ ls /usr/lib/virtualbox/ | grep VBoxPython
VBoxPython2_7.so
VBoxPython3_7m.so
VBoxPython.so
These .so files are installed via the virtualbox package. However, I cannot manage to have neither VBoxPython3.so nor VBoxPython3.8.so by reinstalling. I have looked at the virtualbox code and it seems that the c macro PY_VERSION_HEX indicates 3.7 instead of 3.8, which would be caused by the wrong Python.h header being included.
How can I force virtualbox package to include /usr/include/python3.8/Python.h instead of /usr/include/python3.7/Python.h ?
Uninstalling python3.7 is not an option.
The pyvbox package docs (redirected from pyvbox) imply that the .so files aren't installed by it. But, they do note that the Python package is dependent on the VirtualBox SDK. Which, in turn, recommend installing the SDK to the system Python manually. That latest seems irky, especially when done outside a package manager but maybe they know better.
Digging around, I found the .so files are owned by the OS package virtualbox. In other words, virtualbox must be installed with Python3.8 support.
# Who owns/provides this file (use your dist equivalent or duckduckgo)
$ yay -Fy /usr/lib/virtualbox/VBoxPython3_8.so
...
usr/lib/virtualbox/VBoxPython3_8.so is owned by community/virtualbox 6.1.6-1
I suggest you reinstall the virtualbox package for your distribution. Installing the latest version will provide the latest .so library files Virtualbox supports.
Let's test the theory
# Install Virtualbox and the Virtualbox SDK
yay -S virtualbox virtualbox-sdk python-virtualbox
# Assert we aren't getting an error
$ python -c "import virtualbox; _vbox = virtualbox.VirtualBox(); print('Success!')"
Success!
Otherwise, if the package doesn't include 3.8 support, you'll have to compile it yourself while making sure LD_LIBRARY_PATH points to your Python 3.8 dev libraries. But, I strongly recommend against this. It is infinitely better to work along with your package manager and benefit from upstream work.
Good luck!
I'm making a program using Python2.7 and Kivy1.9.2-dev, and trying to package it with PyInstaller-3.0 for different systems as a single executable.
The systems I'm trying to package it for are these:
64-bit Linux Mint 17.3
32-bit Linux Mint 17 (also tried while upgrading to 17.1 and 17.3)
32-bit Windows XP SP3
Raspbian (Raspberry Pi)
On all these systems the program is working well when just run with Python, uncompiled. (so, all Kivy dependencies are fine, too).
However, out of the executables made with PyInstaller, only the one made on 64-bit Linux works as one file. The Windows and Raspbian executables mostly work (I'll write about it later), but the one made on 32-bit Linux still doesn't run. It gives the following error when run (I tried running it on both 32 and 64-bit Linux):
Traceback (most recent call last):
File "<string>", line 11, in <module>
File "/media/Data/Programming/Python/installers/PyInstaller-3.0/PyInstaller/loader/pyimod03_importers.py", line 363, in load_module
exec(bytecode, module.__dict__)
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 48, in <module>
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/extern/__init__.py", line 60, in load_module
ImportError: The 'six' package is required; normally this is bundled with this package so if you get this warning, consult the packager of your distribution.
pyi_rth_pkgres returned -1
Here's what I'm confused about:
Inside my "/usr/local/lib/python2.7/dist-packages/" there is no "pkg_resources" folder, but the above indicates that it, apparently, is there... It even reads the files there successfully. Is it something that gets created just when the executable starts?
I don't specifically use "six" for anything, before this error I didn't even know it existed.
"Six" IS installed on my system, as confirmed by the Package Manager and by Pip. It's located in "/usr/local/lib/python2.7/dist-packages/". I guess PyInstaller can't find it for some reason (as, I believe, if everything is packaged correctly it doesn't have to be there on the system where the executable is run.)
So, my question is pretty typical, what may cause this problem for PyInstaller (just in case, I DID run the "setup.py install" for it), and how to get around/fix it?
Thanks!
PS: On a side note, I mentioned the problems with Windows and Raspbian executables. On Windows, the exe only runs when there is "zlib1.dll" present in the same folder (even if specifically packaged into the exe with PyInstaller, it doesn't work), and on Raspbian I only got the program working without using "--onefile" (with "--onefile", it seems PyInstaller doesn't package any Python binaries into the executable, like libpython.2.7.so, and maybe others too)
Well, adding 'six' into hidden packages, as Clement suggested, didn't work, but started a sequence of trial-and-error that finally led to a solution.
After the test with "hiddenimports" didn't work I tried just importing 'six' into my Python code. And the compiled executable no longer showed this error! However, it now said that the package named 'packaging' is required... Which I didn't have installed.
To put it short, starting from the initial problem, I did this:
Installed 'packaging' using 'pip':
sudo pip install packaging
Added these imports into my main Python code:
import six
import packaging
import packaging.version
import packaging.specifiers
(all the imports added were trial-and-error, done until the PyInstaller-made executable finally worked).
Seems a bit hack-y, as making the executable for a 64-bit Linux didn't require any of these imports, but at least it now works, and the executable size is basically unaffected.
For the following setup (anaconda):
PyInstaller: 3.2
Python: 3.5.2
Platform: Windows-10-10.0.10240-SP0
Numpy: 1.11.1
And the following mwe.py:
import numpy
print ("hello world")
I had to do the following to fix:
pip install packaging
Build with the following bat file (^ is the BAT line continuation):
pyinstaller --noconfirm ^
--hidden-import six ^
--hidden-import packaging ^
--hidden-import packaging.version ^
--hidden-import packaging.specifiers ^
--hidden-import packaging.requirements ^
mwe.py
I had a similar problem. Try to add "six" as well as "kivy" to the hidden_packages in your spec file. If it doesn't work, make sure that setuptools is installed in its 19.2 version. It seemed to be the problem for me on Windows. Hope it helps.
I'm having a python-3 project that uses the plot-directive to generate and embed matplotlib's diagrams on the fly, and i'm using ReadTheDocs for auto-generating the project's documentation.
The plot-directive indeed works ok in python-2, but it currently fails in python-3.
Specifically the failure i'm getting on the logs of RTD is this:
Build Standard Error
html
-----
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/wltp/envs/master/lib/python3.4/site-packages/sphinx/application.py", line 325, in setup_extension
mod = __import__(extension, None, None, ['setup'])
ImportError: No module named 'matplotlib'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/wltp/envs/master/lib/python3.4/site-packages/sphinx/cmdline.py", line 253, in main
warningiserror, tags, verbosity, parallel)
File "/home/docs/checkouts/readthedocs.org/user_builds/wltp/envs/master/lib/python3.4/site-packages/sphinx/application.py", line 119, in __init__
self.setup_extension(extension)
File "/home/docs/checkouts/readthedocs.org/user_builds/wltp/envs/master/lib/python3.4/site-packages/sphinx/application.py", line 328, in setup_extension
err)
sphinx.errors.ExtensionError: Could not import extension matplotlib.sphinxext.plot_directive (exception: No module named 'matplotlib')
Extension error:
Could not import extension matplotlib.sphinxext.plot_directive (exception: No module named 'matplotlib')
And the culprit can be traced to matplotlib not being compiled due to mnissing 'freetype' C lib:
Setup Output
...
requirements
-----
...
BUILDING MATPLOTLIB
matplotlib: yes [1.4.2]
python: yes [3.4.0 (default, Apr 11 2014, 13:05:11) [GCC
4.8.2]]
platform: yes [linux]
REQUIRED DEPENDENCIES AND EXTENSIONS
numpy: yes [not found. pip may install it below.]
six: yes [six was not found.]
dateutil: yes [dateutil was not found. It is required for date
axis support. pip/easy_install may attempt to
install it after matplotlib.]
pytz: yes [pytz was not found. pip will attempt to install
it after matplotlib.]
tornado: yes [tornado was not found. It is required for the
WebAgg backend. pip/easy_install may attempt to
install it after matplotlib.]
pyparsing: yes [pyparsing was not found. It is required for
mathtext support. pip/easy_install may attempt to
install it after matplotlib.]
pycxx: yes [Official versions of PyCXX are not compatible
with matplotlib on Python 3.x, since they lack
support for the buffer object. Using local copy]
libagg: yes [pkg-config information for 'libagg' could not
be found. Using local copy.]
freetype: no [The C/C++ header for freetype2 (ft2build.h)
could not be found. You may need to install the
development package.]
OPTIONAL LATEX DEPENDENCIES
dvipng: yes [version 1.14]
ghostscript: yes [version 9.10]
latex: yes [version 3.1415926]
pdftops: no
============================================================================
* The following required packages can not be built:
* freetype
To make the doc-generation pass, I'm forced to "disable" plot-directive by mocking it out, as instructed in the RTD FAQ, using the following code in the ./conf.py file.
I've tried with various combinations of virtualenv (with or without site-package visibility), rtd-specific requirements.txt, with no success.
Has anybody found a way to do it?
These are some hints for those willing to dig further into the issue:
The "official" list of pre-installed native libraries on RTD
The actual pip-requirements.txt file used to setup the build-server, as found from RTD's sources.
Since today, the problem is officially solved, according to rtfd issue #896.
All required dependencies (matplotlib, scipy and numpy) are also installed for python-3, so mocking is not necessary anymore.
To make use of it, in the Advanced settings make the following choices:
Check: Install Project: Install your project inside a virtualenv using setup.py install
Select: Python interpreter: CPython 3.x
Check: Use system packages: Give the virtual environment access to the global site-packages dir.
...although with a little training, it is easy to write python-2/3 compatible code.
I have this same problem and it seems to have to do with the RTD interpreter. My project is in Python 3, and I selected as much in RTD Settings and it fails (can't find matplotlib). I was able to work around it by using selecting the Python 2.x interpreter (in Advanced Settings; also allow the global site packages), as opposed to Python 3.x. I use a RTD-specific requirements file (which only has what I need beyond the global site packages: sphinxcontrib-napoleon). Matplotlib is now seen, but now I have issues from Python 2 backwards compatibility. But at least that issue is sorta solved.
I am experiencing a problem when using virtualenv in openSuSE 12.3 with Python3:
I installed the python3 and python3-devel packages.
Then I installed the newest distribute and pip and finally virtualenv using pip.
When I try to create a virtualenv I get the following error:
$ virtualenv-3.3 venv01
Using base prefix '/usr'
New python executable in venv01/bin/python3.3
Also creating executable in venv01/bin/python
Installing distribute.........................................................................................................................................................................................................................................................................................................................................................................................................done.
Installing pip....
Complete output from command /home/user/venv01/bin/python3.3 -x /home/user/venv01/bin/easy_install /usr/local/lib/pytho...ort/pip-1.3.1.tar.gz:
/home/user/venv01/bin/python3.3: can't open file '/home/user/venv01/bin/easy_install': [Errno 2] No such file or directory
----------------------------------------
...Installing pip...done.
Traceback (most recent call last):
File "/usr/local/bin/virtualenv-3.3", line 9, in <module>
load_entry_point('virtualenv==1.9.1', 'console_scripts', 'virtualenv-3.3')()
File "/usr/local/lib/python3.3/site-packages/virtualenv.py", line 979, in main
no_pip=options.no_pip)
File "/usr/local/lib/python3.3/site-packages/virtualenv.py", line 1094, in create_environment
install_pip(py_executable, search_dirs=search_dirs, never_download=never_download)
File "/usr/local/lib/python3.3/site-packages/virtualenv.py", line 667, in install_pip
filter_stdout=_filter_setup)
File "/usr/local/lib/python3.3/site-packages/virtualenv.py", line 1057, in call_subprocess
% (cmd_desc, proc.returncode))
OSError: Command /home/user/venv01/bin/python3.3 -x /home/user/venv01/bin/easy_install /usr/local/lib/pytho...ort/pip-1.3.1.tar.gz failed with error code 2
Running it with -vvv yields some interesting output that shows that distribute
is extracted to <venv>/local/lib/python3.3/site-packages/.
I wonder why distribute is not installed into <venv>/lib/python3.3/site-packages/?
Does anyone have an idea why this happens and how I can use virtualenv in
openSuSE without compiling python3 myself?
[now fixed in latest patches from opensuse]
[oh! i just realised you are the same person as the original link. sorry. but i will leave this as it is a top result for google search on this issue (was searching myself for any update) so it may help others.]
this is a known issue, discussed at https://forums.opensuse.org/english/get-technical-help-here/applications/484475-using-virtualenv-python-3-a.html and with an open bug at https://bugzilla.novell.com/show_bug.cgi?id=809831
the only work-around i know of is to install everything yourself. it's not so hard, and i describe what is necessary at http://www.acooke.org/cute/GettingPyt0.html
basically:
install python 3.3 from source (do an "altinstall" to install as /usr/local/bin/python3.3)
fix the lib issue (link lib_dynload from /usr/local/lib64/python3.3 to /usr/local/lib/python3.3)
install distutils
install virtualenv
then you can use python3.3 and virtual-env-3.3 etc as expected.
Recently I upgraded my Ubuntu to version 11.10 so I had to reinstall python3 modules. I downloaded latest PyQt4 but when I run configure.py I get following error:
Qt Designer plugin disabled because Python library couldn't be found
An internal error occured. Please report all the output from the program,
including the following traceback, to support#riverbankcomputing.com.
Traceback (most recent call last):
File "configure.py", line 2269, in <module>
main()
File "configure.py", line 2254, in main
subdirs=pyqt.qpy_libs() + pyqt_modules + xtra_modules + pyqt.tools(),
File "configure.py", line 957, in tools
link = "%s -lpython%d.%d%s" % (lib_dir_flag, py_major, py_minor, abi)
UnboundLocalError: local variable 'lib_dir_flag' referenced before assignment
I really haven't got a clue what is wrong. btw. Before I upgraded Ubuntu to 11.10 I used python3.1 and everything was fine.
Make sure you have the correct sip packages installed before you try to build PyQt4. For ubuntu, I think you will need the python-sip-dev and python3-sip-dev packages (plus any dependencies). Alternatively, you could download and build the latest version of sip from source.
When you run the configure.py script for either pyqt or sip, it is essential that you use the correct version of python, e.g:
/usr/bin/python3.2 configure.py