Can cython be compiled with icc? - python-3.x

I am trying to build cython from source with icc compiler on Ubuntu 14.04 as my python is compiled with Intel icc compiler.
When I tried to install cython using pip3 install cython and then ran cython I got following error
Traceback (most recent call last):
File "/usr/local/bin/cython", line 9, in <module>
load_entry_point('Cython==0.24', 'console_scripts', 'cython')()
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 351, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2363, in load_entry_point
return ep.load()
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2088, in load
entry = __import__(self.module_name, globals(),globals(), ['__name__'])
File "/usr/local/lib/python3.4/dist-packages/Cython/Compiler/Main.py", line 28, in <module>
from .Scanning import PyrexScanner, FileSourceDescriptor
ImportError: /usr/local/lib/python3.4/dist-packages/Cython/Compiler/Scanning.cpython-34m.so: undefined symbol: __intel_sse2_strchr
How does one go about installing cython from source using icc compiler ?
I tried this and it does not work
From the cython directory(downloaded from github)
python3.4 setup.py CC=icc
I get the following message -
/home/aa/libPython/cython/Cython/Distutils/build_ext.py:20: UserWarning:
Cython.Distutils.build_ext does not properly handle dependencies and is deprectated. Use Cython.Build.build_ext instead.
"Cython.Distutils.build_ext does not properly handle dependencies "
Unable to find pgen, not compiling formal grammar.
invalid command name 'CC=icc'

CC=icc is not enough to build with icc. You should also link with icc (https://stackoverflow.com/a/10891764/196561; icc will add its internal libraries to ELF file), so find name of linker variable for your setup.py, probably LD and set it to icc too LD=icc (default is probably gcc).
Actually it is LINKCC - https://github.com/cython/cython/blob/970c2fc0e676ca22016e14147ada0edba937dc6b/Cython/Build/BuildExecutable.py
CC = get_config_var('CC', os.environ.get('CC', ''))
CFLAGS = get_config_var('CFLAGS') + ' ' + os.environ.get('CFLAGS', '')
LINKCC = get_config_var('LINKCC', os.environ.get('LINKCC', CC))
So, correct build for cython with icc should be with CC=icc LINKCC=icc, don't know how variables should be passed into setup.py, check How to tell distutils to use gcc? or try
CC=icc LINKCC=icc python3.4 setup.py
Update: According to message from gansub, "LDSHARED=icc" env. variable will help to build cython: "You need to set the environment variable LDSHARED=icc" - https://chat.stackoverflow.com/transcript/message/31231907#31231907 and https://stackoverflow.com/a/37914227
Update from Syrtis Major: There is article "Thread Parallelism in Cython*" https://software.intel.com/en-us/articles/thread-parallelism-in-cython by Nguyen, Loc Q (Intel), December 15, 2016 with recommendation of LDSHARED="icc -shared" CC=icc:
To explicitly use the Intel icc to compile this application, execute the setup.py file with the following command:
$ LDSHARED="icc -shared" CC=icc python setup.py build_ext --inplace

Related

Python3 - Failed to load library on "import cairosvg"

I'm trying to install CairoSVG for Python 3. I ran pip3 install cairosvg without any errors, but when I try to import the module via import cairosvg, I get this error:
Traceback (most recent call last):
File "<pyshell#5>", line 1, in <module>
import cairosvg
File "D:\Programs\Python3\lib\site-packages\cairosvg\__init__.py", line 41, in <module>
from . import surface # noqa
File "D:\Programs\Python3\lib\site-packages\cairosvg\surface.py", line 24, in <module>
import cairocffi as cairo
File "D:\Programs\Python3\lib\site-packages\cairocffi\__init__.py", line 41, in <module>
cairo = dlopen(ffi, 'cairo', 'cairo-2', 'cairo-gobject-2')
File "D:\Programs\Python3\lib\site-packages\cairocffi\__init__.py", line 38, in dlopen
raise OSError("dlopen() failed to load a library: %s" % ' / '.join(names))
OSError: dlopen() failed to load a library: cairo / cairo-2 / cairo-gobject-2
cairosvg python package doesn't install cairo library on your computer. You need to download an installer and make sure it's accessable by python. Also you need to be aware if python is built for the same instruction set as cairo library (e.g. x32 — x32 and x86_64 — x86_64). Python can't use libraries built for other architecture.

RISCV debug tests - 'gdbserver.py' fails for missing module

When I try to run the following in the /riscv-tests/debug directory
it fails.
./gdbserver.py --spike64 --cmd ../../bin/spike
Traceback (most recent call last):
File "./gdbserver.py", line 10, in
import targets
File "/home/dave/MyProjects/RiscV/build/riscv-tools/riscv-tests/debug/targets.py", line 4, in
import testlib
File "/home/dave/MyProjects/RiscV/build/riscv-tools/riscv-tests/debug/testlib.py", line 9, in
import pexpect
ImportError: No module named pexpect
I have searched the entire RISCV installation for "pexpect.py" with no success. I installed the complete RISCV toolchain. I can build code and run
the ISA simulator using spike and pk.
Looks like the problem is with my python installation. I found 'python-pexpect' in my Linux package manager and installed. But I am now having errors with other python system libraries.

ImportError with cx_freeze and pywin32: Module 'pythoncom' isn't in frozen sys.path

I am attempting to create a .exe on Windows 7 from a python3 script using cx_freeze. the Script involves using pywin32 to manipulate Excel files. I can build the .exe successfully from my setup.py file; however, when I run the .exe, the following error is thrown:
Traceback (most recent call last):
File
"C:\Python33\lib\site-packages\cx_Freeze\initscripts\Console3.py",
line 27, in exec(code,m_dict_)
File "MyScript.py", line
12, in < module >
File
"C:\Python\64-bit\3.3\lib\importlib_bootstrap.py", line 1558, in
_find_and_load
File "C:\Python\64-bit\3.3\lib\importlib_bootstrap.py", line 1505, in
_find_and_load_unlocked
File "C:\Python\64-bit\3.3\lib\importlib_bootstrap.py", line 313, in
_call_with_frames_removed
File "C:\Python\64-bit\3.3\lib\importlib_bootstrap.py", line 1558, in
_find_and_load
File "C:\Python\64-bit\3.3\lib\importlib_bootstrap.py", line 1525, in
_find_and_load_unlocked
File "C:\Python33\lib\site-packages\win32com__init__.py", line 6, in
< module>
import pythoncom
File
"C:\Python\64-bit\3.3\lib\importlib_bootstrap.py", line 1558, in
_find_and_load
File "C:\Python\64-bit\3.3\lib\importlib_bootstrap.py", line 1525, in
_find_and_load_unlocked
File "C:\Python33\lib\site-packages\pythoncom.py", line 3, in
pywintypes._import_pywin32_system_module_("pythoncom", globals())
File "C:\Python33\lib\site-packages\win32\lib\pywintypes.py", line 61,
in _import_pywin32_system_module_
raise ImportError("Module '%s'
isn't in frozen sys.path %s" % (modname, sys.path))
ImportError: Module 'pythoncom' isn't in frozen sys.path
['C:\Python33\build\exe.win-amd64\3.3\MyScript.exe',
'C:\Python33\build\exe.win-amd64\3.3',
'C:\Python33\build\exe.win-amd64\3.3\MyScript.zip',
'C:\Python33\build\exe.win-amd64\3.3\library.zip']
Here is what my setup.py file currently looks like:
import sys
from cx_Freeze import setup, Executable
base = None
if sys.platform == 'win32':
base = 'Win32GUI'
includes = []
packages = []
executables = [Executable('MyScript.py', base=base)]
include_files = ['MyFolder1/','MyFolder2/Spreadsheet.xls']
setup(name='My Script',
version='0.1',
description='My Script',
executables=executables,
options = {'build_exe': {'includes':includes,
'packages':packages,
'include_msvcr':True,
'include_files':include_files}})
So far, I have tried listing both 'pythoncom' and 'win32com' in both the includes and the packages lists. Any help is greatly appreciated!
I had exactly the same issue when using pyinstaller to pack py into executable. Tried everything, but find out the issue is hidden in pyinstaller version. I was using pyinstaller 4.7 but when downgrade into 3.6, the exe works flawlessly.
Looking at the code, it looks like you need to ensure that a file called something like pythoncom33.dll is copied into the build directory.
So the whole problem actually stemmed from having the 32-bit version of pywin32 installed while running 64-bit versions of Python-3.3.2 and Windows 7. After adding pythoncom33.dll to the include_files of my setup.py as Thomas K had suggested, I got another error:
ImportError: DLL load failed: %1 is not a valid Win32 application
After some research, I found that this error is typical when mixing 32-bit and 64-bit. So I uninstalled pywin32 32-bit and installed pywin32 64-bit but my script threw yet another error:
import win32api, sys, os
ImportError: DLL load failed: The specified module could not be found.
As suggested in this post, I copied the 28 win32*.pyd files from the Lib/site-packages/win32 folder to the Python33 folder right next to python.exe , and everything worked.
Add this to the top of your main.py:
import pywintypes
import win32api
see:
PyWin32 and Python 3.8.0
So I ran into a similar problem where cx_Freeze was failing to pull in the pywintypes DLL, but frustratingly enough it was only on some machines which compelled me to explore the issue a bit further (I have a bad habit of chasing rabbits down holes). It seems that pywin32 tries to install its DLLs into a Windows system directory, but if that fails due to a lack of privileges it will fallback to placing them in your Python directory instead. The behavior I was seeing was a result of the fact that cx_Freeze applies a blacklist of well-known system folder paths against the collection of dependencies it has discovered in order to avoid pulling in operating system DLLs.
Copying these DLLs as needed by adding a path to them to the "include_files" parameter will indeed address the issue (just remember that you can't guarantee that they will be placed in the system folder). An alternative solution is to override the blacklist with a whitelist which you can configure using the "bin_includes" / "bin_path_includes" parameters. Here is a quick example that shows how to configure this parameter using a setup.py file (for Python 2.7, but the only difference should be the pywintypes DLL filename):
from cx_Freeze import setup, Executable
# This can also be an absolute path to the file if you wish to be very specific
bin_includes = ["pywintypes27.dll"]
options = {
"build_exe": {
"bin_includes": bin_includes
}
}
executable = Executable(script='main.py')
setup(version='x.x.x',
options=options,
executables=[executable])
Update pyinstaller related vision.
python -m pip install pyinstaller==3.6

Scipy not working with python 3 but does with 2.6(default)

I have installed Scipy by build and install from the terminal. But I get the following error when I try to execute the following simple code:
from scipy.stats.mstats_basic import gmean
ab=[1,2,3,4,5,6]
print(gmean(ab))
This is the error I am getting:
Traceback (most recent call last):
File "/home/prashanthb/workspace/rankalgo/rough1.py", line 2, in <module>
from scipy.stats.mstats_basic import gmean
File "/usr/local/lib/python3.1/dist-packages/scipy/stats/__init__.py", line 321, in <module>
File "/usr/local/lib/python3.1/dist-packages/scipy/stats/stats.py", line 194, in <module>
File "/usr/local/lib/python3.1/dist-packages/scipy/special/__init__.py", line 525, in <module>
ImportError: No module named _cephes
But the same code works when I switch to the default interpreter python 2.6. I have installed Libatlas and lapack before and Numpy(already installed under python3) seems to work well.
I have got mpmath installed too.
Question: How to proceed from here to get it to work with python3? If anybody knows any other library to calculate geometric mean please do mention.
Thanks in advance
prashanth
UPDATE: I found this while building setup.py it again:
DeprecationWarning)
customize GnuFCompiler
Could not locate executable g77
Could not locate executable f77
customize IntelFCompiler
Could not locate executable ifort
Could not locate executable ifc
customize LaheyFCompiler
Could not locate executable lf95
customize PGroupFCompiler
Could not locate executable pgfortran
customize AbsoftFCompiler
Could not locate executable f90
customize NAGFCompiler
Found executable /usr/bin/f95
customize VastFCompiler
customize CompaqFCompiler
Could not locate executable fort
customize IntelItaniumFCompiler
Could not locate executable efort
Could not locate executable efc
customize IntelEM64TFCompiler
customize Gnu95FCompiler
Found executable /usr/bin/gfortran
customize Gnu95FCompiler
customize Gnu95FCompiler using config
compiling '_configtest.c':

Python3.2 PyQt4 installation: UnboundLocalError

Recently I upgraded my Ubuntu to version 11.10 so I had to reinstall python3 modules. I downloaded latest PyQt4 but when I run configure.py I get following error:
Qt Designer plugin disabled because Python library couldn't be found
An internal error occured. Please report all the output from the program,
including the following traceback, to support#riverbankcomputing.com.
Traceback (most recent call last):
File "configure.py", line 2269, in <module>
main()
File "configure.py", line 2254, in main
subdirs=pyqt.qpy_libs() + pyqt_modules + xtra_modules + pyqt.tools(),
File "configure.py", line 957, in tools
link = "%s -lpython%d.%d%s" % (lib_dir_flag, py_major, py_minor, abi)
UnboundLocalError: local variable 'lib_dir_flag' referenced before assignment
I really haven't got a clue what is wrong. btw. Before I upgraded Ubuntu to 11.10 I used python3.1 and everything was fine.
Make sure you have the correct sip packages installed before you try to build PyQt4. For ubuntu, I think you will need the python-sip-dev and python3-sip-dev packages (plus any dependencies). Alternatively, you could download and build the latest version of sip from source.
When you run the configure.py script for either pyqt or sip, it is essential that you use the correct version of python, e.g:
/usr/bin/python3.2 configure.py

Resources