I am trying to set up visual studio projects with debug and release builds in a large SCons project. The manual proclaims
Multiple calls to MSVSProject with different variants are allowed; all variants will be added to the project file with their appropriate build targets and sources.
However, when I try to do this, I get the error that "Multiple ways to build the same target were specified".
Minimal example (SConstruct file)
import os
from SCons.Script import *
env = Environment()
for variant in ['debug', 'release']:
env.MSVSProject(
target = 'hello' + env['MSVSPROJECTSUFFIX'],
srcs = 'hello.cpp',
buildtarget = os.path.join(variant, 'hello.exe'),
variant = variant)
This gives the following output:
scons: Reading SConscript files ...
scons: warning: Two different environments were specified for target hello.vcxproj,
but they appear to have the same action: GenerateProject(target, source, env)
File "SConstruct", line 7, in <module>
scons: *** Multiple ways to build the same target were specified for: hello.vcxproj (from ['prj_inputs:"python.exe" -c "from os.path import join; import sys; sys.path = [ join(sys.prefix, \'Lib\', \'site-packages\', \'scons-3.1.2\'), join(sys.prefix, \'scons-3.1.2\'), join(sys.prefix, \'Lib\', \'site-packages\', \'scons\'), join(sys.prefix, \'scons\') ] + sys.path; import SCons.Script; SCons.Script.main()" -C "." -f SConstructutf-8; ppdefs: incpath: "debug\\hello.exe" "debug" "hello.cpp "hello.vcxproj"'] and from ['prj_inputs:"python.exe" -c "from os.path import join; import sys; sys.path = [ join(sys.prefix, \'Lib\', \'site-packages\', \'scons-3.1.2\'), join(sys.prefix, \'scons-3.1.2\'), join(sys.prefix, \'Lib\', \'site-packages\', \'scons\'), join(sys.prefix, \'scons\') ] + sys.path; import SCons.Script; SCons.Script.main()" -C "." -f SConstructutf-8; ppdefs: incpath: "release\\hello.exe" "release" "hello.cpp "hello.vcxproj"'])
File "SConstruct", line 7, in <module>
Is this a bug in SCons, or am I not understanding how this is supposed to work?
I realize that the variants can be passed in a list as a single call, but maintaining seperate lists of sources and targets for different build types would not be possible without a major rewrite of the existing build infrastructure. I was hoping to use one environment per build type, and the quote from the manual makes it seem like SCons should be able to combine them into one MSVS project.
Any help would be greatly appreciated!
This is a bug in SCons, and has been submitted
Related
I'm trying to build GStreamer on windows using gst-build.
My environment:
Visual studio 2019 Professional
Python 3.9 64 bit
Meson 0.58.999
Ninja 1.10.2
the error message is:
>ninja -C build
ninja: Entering directory `build'
[1/30] Generating gir-glib with a custom command (wrapped by meson to set PATH)
FAILED: subprojects/gobject-introspection/gir/GLib-2.0.gir
[long list of python subprocess args with include patsh, c files etc]
Traceback (most recent call last):
File "C:\Work\GStreamer-build\source\gst-build\build\subprojects\gobject-introspection\tools\g-ir-scanner", line 98, in <module>
from giscanner.scannermain import scanner_main
File "C:\Work\GStreamer-build\source\gst-build\build\subprojects\gobject-introspection\giscanner\scannermain.py", line 35, in <module>
from giscanner.ast import Include, Namespace
File "C:\Work\GStreamer-build\source\gst-build\build\subprojects\gobject-introspection\giscanner\ast.py", line 29, in <module>
from .sourcescanner import CTYPE_TYPEDEF, CSYMBOL_TYPE_TYPEDEF
File "C:\Work\GStreamer-build\source\gst-build\build\subprojects\gobject-introspection\giscanner\sourcescanner.py", line 35, in <module>
from giscanner._giscanner import SourceScanner as CSourceScanner
ImportError: DLL load failed while importing _giscanner: The specified module could not be found.
ninja: build stopped: subcommand failed.
if I check the build directory:
So the file does build, but for some reason can't be imported.
I've seen from here that there were some changes to python 3.8 in how DLLs are searched for, but even if I manually open a python prompt in the parent directory and add both the parent and the giscanner directories to sys.path and using os.add_dll_directory I still can't import the module:
>>> import os,sys
>>> a = os.add_dll_directory(os.getcwd())
>>> a
<AddedDllDirectory('C:\\Work\\GStreamer-build\\source\\gst-build\\build\\subprojects\\gobject-introspection')>
>>> b = os.add_dll_directory(os.path.join(os.getcwd(),'giscanner'))
>>> b
<AddedDllDirectory('C:\\Work\\GStreamer-build\\source\\gst-build\\build\\subprojects\\gobject-introspection\\giscanner')>
>>> sys.path.insert(0, os.getcwd())
>>> sys.path.insert(0, os.path.join(os.getcwd(),'giscanner'))
further digging shows that pkg-config doesn't seem to know where to find gio, however that was also built sucessfully as part of the build so not sure why it doesn't know.
Unsure the actual problem, but the workaround is to revert Python to 3.7.9 (64 bit)
https://www.python.org/downloads/release/python-379
Its also been suggested you can build without that component using
-Dintrospection=disabled
as of today, I have been able to compile my codes with Cython and speed up my programs. Now, I would like to learn parallel computing but since I am new to this I am trying simple examples first and I was trying to implement the one given here. When I compile my setup.py using the following code:
from distutils.core import setup
from Cython.Build import cythonize
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules = [
Extension('test_multi',
['test_multi.pyx'],
libraries=['m'],
extra_compile_args = ['-O3', '-ffast-math', '-march=native', '-fopenmp' ],
extra_link_args=['-fopenmp']
)
]
setup(cmdclass={"build_ext": build_ext}, ext_modules=ext_modules)
the compiler is warning me that it does not recognize '-O3', '-ffast-math', '-march=native', '-fopenmp' and the compile command is failing. I also tried to modify ext_modules as defined here:
ext_modules = [
Extension(
'test_multi',
['test_multi.pyx'],
extra_compile_args=['/openmp'],
extra_link_args=['/openmp'],
)
]
but also in this case, '/openmp' is not recognized.
What am I missing? I tried to have a look around but I was not able to find an answer (or at least if there was one, I did not get it). Could you please help me? If you need some extra info, please ask me. Thanks a lot!
EDIT: I am using Windows 10 and Visual Studio Build Tools 2019.
EDIT 2: the warning I get is LNK4044:unrecognized option '/openmp'; ignored.
The goal is to use the pytest unit test framework for a Python3 project that uses Cython. This is not a plug-and-play thing, because pytest by default is not able to import the Cython modules.
One unsuccessful solution would be to use the pytest-cython plugin, but it simply does not work for me:
> py.test --doctest-cython
usage: py.test [options] [file_or_dir] [file_or_dir] [...]
py.test: error: unrecognized arguments: --doctest-cython
inifile: None
rootdir: /censored/path/to/my/project/dir
To verify that I have the package installed:
> pip freeze | grep pytest-cython
pytest-cython==0.1.0
UPDATE:
I'm using PyCharm and it seems that it is not using my pip-installed packages but rather uses a custom(?) pycharm repository for packages used by my project. Once I added pytest-cython to that repository, the command runs but strange enough it doesn't recognize the Cython module anyway, although the package/add-on is specifically designed for that purpose:
> pytest --doctest-cython
Traceback:
tests/test_prism.py:2: in <module>
from cpc_naive.prism import readSequence, processInput
cpc_naive/prism.py:5: in <module>
from calculateScore import calculateScore, filterSortAlphas,
calculateAlphaMatrix_c#, incrementOverlapRanges # cython code
E ImportError: No module named 'calculateScore'
Another unsuccessful solution I got here is to use pytest-runner, but this yields:
> python3 setup.py pytest
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'pytest'
UPDATE:
I first had forgotten to add setup_requires=['pytest-runner', ...] and tests_require=['pytest', ...] to the setup script. Once i did that, I got another error:
> python3 setup.py pytest
Traceback (most recent call last):
File "setup.py", line 42, in <module>
tests_require=['pytest']
(...)
AttributeError: type object 'test' has no attribute 'install_dists'
UPDATE 2 (setup.py):
from distutils.core import setup
from distutils.extension import Extension
from setuptools import find_packages
from Cython.Build import cythonize
import numpy
try: # try to build the .c file
from Cython.Distutils import build_ext
except ImportError: # if the end-user doesn't have Cython that's OK; you should have shipped the .c files anyway.
use_cython = False
else:
use_cython = True
cmdclass = {}
ext_modules = []
if use_cython:
ext_modules += [
Extension("cpc_naive.calculateScore", ["cpc_naive/calculateScore.pyx"],
extra_compile_args=['-g'], # -g for debugging
define_macros=[('CYTHON_TRACE', '1')]),
]
cmdclass.update({'build_ext': build_ext})
else:
ext_modules += [
Extension("cpc_naive.calculateScore", ["cpc_naive/calculateScore.c"],
define_macros=[('CYTHON_TRACE', '1')]), # compiled C files are stored in /home/pdiracdelta/.pyxbld/
]
setup(
name='cpc_naive',
author=censored,
author_email=censored,
license=censored,
packages=find_packages(),
cmdclass=cmdclass,
ext_modules=ext_modules,
install_requires=['Cython', 'numpy'],
include_dirs=[numpy.get_include()],
setup_requires=['pytest-runner'],
tests_require=['pytest']
)
UPDATE 3 (partial fix):
As suggested by #hoefling I downgraded pytest-runner to a version <4 (in fact 3.0.1) and this resolves the error in update 1, but now I get the same Exception as with the pytest-cython solution:
E ImportError: No module named 'calculateScore'
It just doesn't seem to recognize the module. Perhaps this is due to some absolute/relative import mojo I don't understand.
How can I use pytest with Cython? How can I discover why these methods aren't working and then fix it?
FINAL UPDATE:
After taking both the original problem and the question Updates into consideration (thanks #hoefling for solving these issues!), this question is now reduced to the question of:
why can pytest no import the Cython module calculateScore, even though running the code just with python (no pytest) works just fine?
As #hoefling suggested, one should use pytest-runner version <0.4 to avoid the
AttributeError: type object 'test' has no attribute 'install_dists'
To then answer the actual and final question (in addition to partial, off-topic, user-specific fixes added to the question post itself) of why pytest cannot import the Cython module calculateScore, even though running the code just with python (no pytest) works just fine:
that remaining issue is solved here.
I know there are lots of similar questions on the website but I couldn't find an answer to my problem.
I'm wrapping C++ classes with Cython in order to use them with Python3. After building the external module with a setup.py, when I run the python program I got the following error:
from "name of.pyx file" import "name of the class to import"
Import error: /home/.../filename.so: undefined symbol: _ZTINSt8ios_base7failureB5cxx11E.
I'm on Ubuntu 16.04, I build the extensions from the terminal with the command line python3 setup.py build_ext --inplace, and then run the .py from the terminal or from Spyder in Anaconda (I got the error in both cases.)
From what I read the error might come from the cython compilation because i'm not linking some libraries. Is this true? If it is, could someone explain me how to do it?
I let you here my setup.py, in comments all the different setups I tried.
setup.py
from distutils.core import setup, Extension
from Cython.Build import cythonize
import numpy
#setup(ext_modules = cythonize(
#"pycoralv1.pyx", # our Cython source
#sources=["coralv1cpp.cpp"], # additional source file(s)
#language="c++", # generate C++ code
#))
#setup(ext_modules = cythonize(Extension(
# "pyCoralv1", # the extension name
# sources=["pyCoralv1.pyx", "Coralv1cpp.cpp"], # the Cython source and
# additional C++ source files
# language="c++", # generate and compile C++ code
# )))
#setup(
# name = "testcoral",
# ext_modules = cythonize('*.pyx'),
#)
ext_modules = [
Extension(
"pyCoralv1",
sources=["pyCoralv1.pyx", "Coralv1cpp.cpp"],
extra_compile_args=['-fopenmp',"-fPIC"],
extra_link_args=['-fopenmp',"-I", "/usr/include/glib-2.0", "-l", "glib-2.0", "-I", "/usr/lib/x86_64-linux-gnu/glib-2.0/include"],
language="c++",
)
]
for e in ext_modules:
e.pyrex_directives = {"boundscheck": False}
setup(
name='Coral library',
ext_modules=cythonize(ext_modules),
include_dirs = [numpy.get_include()]
)
The problem was solved after installing libgcc in anaconda: conda install libgcc, there was a missing library.
I have a little problem with cx_freeze and hoping one of you can help me. I have searched trough this wonderfull forum but I can't find the answer.
I have used cx_freeze before with python 3.3 and ktinker and that worked flawless.
Now I made a little tool with a bit more complex gui and tried Glade.
Building the gui with Glade works perfect for me and on Linux and Windows 7 the application I have made works fine (in python interpreter).
When I run python setup.py bdist_msi I don't see any faults but when I try to run the exe in windows I get this error window:
(I can't post images jet)
The last 4 lines are:
_load_backward_compatible
File "ExtentionLoader_gi_gi.py", line 22, in <module
File "ExtentionLoader_gi_gi.py", line 14, in_bootstrap_
ImportError: DLL load failed: The specified module could not be found
I don't use any plugins, exotic imports so the dll's I have to load are only the dll's for Gobject. The setup file I have made from an example on this forum. For my ktinker app I did not have to import any dll.
Finally the question: Is there a list of dll's somewhere that tells me what dll's I have to add?
And is there something wrong with my setup.py?
The code is nothing special but if you want to check it: https://github.com/EddenBeer/CodeGenerator
The imports in Python:
import csv import sys import datetime
from gi.repository import Gtk
Installed on Windows 7:
Python-3.4.2
cx_Freeze-4.3.3.win32-py3.4
pygi-aio-3.14.0_rev6-setup
Setup.py:
import os, site, sys
from cx_Freeze import setup, Executable
## Get the site-package folder, not everybody will install
## Python into C:\PythonXX
site_dir = site.getsitepackages()[1]
include_dll_path = os.path.join(site_dir, "gnome")
## Collect the list of missing dll when cx_freeze builds the app
missing_dll = ['libgtk-3-0.dll',
'libgdk-3-0.dll',
'libatk-1.0-0.dll',
'libcairo-gobject-2.dll',
'libgdk_pixbuf-2.0-0.dll',
'libjpeg-8.dll',
'libpango-1.0-0.dll',
'libpangocairo-1.0-0.dll',
'libpangoft2-1.0-0.dll',
'libpangowin32-1.0-0.dll',
'libgnutls-26.dll',
'libgcrypt-11.dll',
'libp11-kit-0.dll'
]
## We also need to add the glade folder, cx_freeze will walk
## into it and copy all the necessary files
glade_folder = 'glade'
## We need to add all the libraries too (for themes, etc..)
gtk_libs = ['etc', 'lib', 'share']
## Create the list of includes as cx_freeze likes
include_files = []
for dll in missing_dll:
include_files.append((os.path.join(include_dll_path, dll), dll))
## Let's add glade folder and files
include_files.append((glade_folder, glade_folder))
## Let's add gtk libraries folders and files
for lib in gtk_libs:
include_files.append((os.path.join(include_dll_path, lib), lib))
base = None
## Lets not open the console while running the app
if sys.platform == "win32":
base = "Win32GUI"
executables = [
Executable("CodeGenerator.py",
base=base
)
]
buildOptions = dict(
compressed = False,
includes = ["gi", "csv", "datetime",],
packages = ["gi"],
include_files = include_files
)
setup(
name = "Code Generator",
author = "Ed den Beer",
version = "1.0",
description = "Generating copy instructions for RsLogix5000 out of a list with tags in a CSV file",
options = dict(build_exe = buildOptions),
executables = executables
)
My problem is solved.
Looking for an answer if found a utility called ListDlls.exe.
In this link is explaned how to use it:
https://bitbucket.org/anthony_tuininga/cx_freeze/issue/92/pygi-and-cx_freeze-error