I'm trying to build a lib without the test files, as described here https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html. In the pyproject.toml, I got:
[tool.setuptools.packages.find]
include = ['lib*']
exclude = ['^.*tests']
with this exclude pattern correctly matching the 3 last elements:
lib
lib.mod1
lib.mod2
lib.utils
lib.mod1.tests
lib.mod2.tests
lib.utils.tests
Yet, when I build with python -m build and install the resulting wheel, I still get the 3 test modules in the distribution. What am I doing wrong?
It looks like you're using regex syntax. But this is a glob/wildmatch pattern, not regex.
Try this:
[tool.setuptools.packages.find]
include = ['lib*']
exclude = ['lib*tests']
Apart from the unsupported regex syntax (see https://stackoverflow.com/a/72524001/1551810), exclude doesn't work if the test files are included in the source distribution (sdist), e.g. because there is a MANIFEST.in that includes them or because of the use of the setuptools-scm package (see also https://github.com/pypa/setuptools/issues/3260).
If the tests are to be excluded from the wheel, but included in the sdist, the solution is to explicitly disable include-package-data (the default is true):
[tool.setuptools]
packages.find.include = ['lib*']
packages.find.exclude = ['lib*tests']
include-package-data = false
Related
I need to do purify build using SCons. Normally in nmake we would do
Nmake install -purify_7.0.1.0-008
How to do the same in SCons? Do I need to add any change in SConstruct file?
You'll need to prefix SHCC,CC, SHCXX, CXX, SHLINK, LINK with purify, and likely add the path to your purify binary to env['ENV']['PATH']
You'll need to do this anywhere (in the main SConstruct, all SConscripts, and if you have any code in site_scons) which creates an Environment() to use.
If you create one Enviroment() and that gets exported to your SConscripts where it's then env.Clone()'d, then you'd only have to change it in the initial env=Environment()
Here's an example (not tested):
env=Environment() # this is your existing code
for v in ['SHCC','CC','SHCXX','CXX','LINK','SHLINK']:
orig_value = env.subst('$%s'%v)
env['v'] = "purify "+ orig_value
print("Changing $%s from :\n%s\nto:\n%s",v, orig_value, env['v'])
I guess this would be a generic question on including libraries with existing makefiles within cmake; but here's my context -
I'm trying to include scintilla in another CMake project, and I have the following problem:
On Linux, scintilla has a makefile in (say) the ${CMAKE_CURRENT_SOURCE_DIR}/scintilla/gtk directory; if you run make in that directory (as usual), you get a ${CMAKE_CURRENT_SOURCE_DIR}/scintilla/bin/scintilla.a file - which (I guess) is the static library.
Now, if I'd try to use cmake's ADD_LIBRARY, I'd have to manually specify the sources of scintilla within cmake - and I'd rather not mess with that, given I already have a makefile. So, I'd rather call the usual scintilla make - and then instruct CMAKE to somehow refer to the resulting scintilla.a. (I guess that this then would not ensure cross-platform compatibility - but note that currently cross-platform is not an issue for me; I'd just like to build scintilla as part of this project that already uses cmake, only within Linux)
So, I've tried a bit with this:
ADD_CUSTOM_COMMAND(
OUTPUT scintilla.a
COMMAND ${CMAKE_MAKE_PROGRAM}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/scintilla/gtk
COMMENT "Original scintilla makefile target" )
... but then, add_custom_command adds a "target with no output"; so I'm trying several approach to build upon that, all of which fail (errors given as comment):
ADD_CUSTOM_TARGET(scintilla STATIC DEPENDS scintilla.a) # Target "scintilla" of type UTILITY may not be linked into another target.
ADD_LIBRARY(scintilla STATIC DEPENDS scintilla.a) # Cannot find source file "DEPENDS".
ADD_LIBRARY(scintilla STATIC) # You have called ADD_LIBRARY for library scintilla without any source files.
ADD_DEPENDENCIES(scintilla scintilla.a)
I'm obviously quote a noob with cmake - so, is it possible at all to have cmake run a pre-existing makefile, and "capture" its output library file, such that other components of the cmake project can link against it?
Many thanks for any answers,
Cheers!
EDIT: possible duplicate: CMake: how do i depend on output from a custom target? - Stack Overflow - however, here the breakage seems to be due to the need to specifically have a library that the rest of the cmake project would recognize...
Another related: cmake - adding a custom command with the file name as a target - Stack Overflow; however, it specifically builds an executable from source files (which I wanted to avoid)..
You could also use imported targets and a custom target like this:
# set the output destination
set(SCINTILLA_LIBRARY ${CMAKE_CURRENT_SOURCE_DIR}/scintilla/gtk/scintilla.a)
# create a custom target called build_scintilla that is part of ALL
# and will run each time you type make
add_custom_target(build_scintilla ALL
COMMAND ${CMAKE_MAKE_PROGRAM}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/scintilla/gtk
COMMENT "Original scintilla makefile target")
# now create an imported static target
add_library(scintilla STATIC IMPORTED)
# Import target "scintilla" for configuration ""
set_property(TARGET scintilla APPEND PROPERTY IMPORTED_CONFIGURATIONS NOCONFIG)
set_target_properties(scintilla PROPERTIES
IMPORTED_LOCATION_NOCONFIG "${SCINTILLA_LIBRARY}")
# now you can use scintilla as if it were a regular cmake built target in your project
add_dependencies(scintilla build_scintilla)
add_executable(foo foo.c)
target_link_libraries(foo scintilla)
# note, this will only work on linux/unix platforms, also it does building
# in the source tree which is also sort of bad style and keeps out of source
# builds from working.
OK, I think I have it somewhat; basically, in the CMakeLists.txt that build scintilla, I used this only:
ADD_CUSTOM_TARGET(
scintilla.a ALL
COMMAND ${CMAKE_MAKE_PROGRAM}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/scintilla/gtk
COMMENT "Original scintilla makefile target" )
... and then, the slightly more complicated part, was to find the correct cmake file elsewhere in the project, where the ${PROJECT_NAME} was defined - so as to add a dependency:
ADD_DEPENDENCIES(${PROJECT_NAME} scintilla.a)
... and finally, the library needs to be linked.
Note that in the commands heretofore, the scintilla.a is merely a name/label/identifier/string (so it could be anything else, like scintilla--a or something); but for linking - the full path to the actual `scintilla.a file is needed (which in this project ends up in a variable ${SCINTILLA_LIBRARY}). In this project, the linking basically occurs through a form of a
list(APPEND PROJ_LIBRARIES ${SCINTILLA_LIBRARY} )
... and I don't really know how cmake handles the actual linking afterwards (but it seems to work)
For consistency, I tried to use ${SCINTILLA_LIBRARY} instead of scintilla.a as identifier in the ADD_CUSTOM_TARGET, but got error: "Target names may not contain a slash. Use ADD_CUSTOM_COMMAND to generate files". So probably this could be solved smarter/more correct with ADD_CUSTOM_COMMAND - however, I read that it "defines a new command that can be executed during the build process. The outputs named should be listed as source files in the target for which they are to be generated."... And by now I'm totally confused so as to what is a file, what is a label, and what is a target - so I think I'll leave at this (and not fix it if it ain't broken :) )
Well, it'd still be nice to know a more correct way to do this eventually,
Cheers!
I just started learning to use SCons, anticipating that it solves some of my issues with make. I'm creating a source hierarchy to understand the basics of SCons.
Let's start with this folder structure:
test/foo: contains main.cpp, main.h
test/bar: contains its own main.cpp, main.h
test/common: contains utils.cpp and utils.h used by both foo and bar
test/external/moo: the source to some external library, containing 'configure' which produces 'Makefile' (not using SCons), so SCons needs to invoke 'make' after 'configure'; I suspect this part might be tricky when build dirs are used
test/build/debug: build dir for debug
test/build/release: build dir for release
Here's what I'd like to do:
Have two types of builds: debug/release where the only difference is that debug specifies -DDEBUG to g++
Use build dirs so that no .o files are created in my source tree. Let's call these build dirs "build/debug" and "build/release"
Be able to invoke ./configure and make on another project that does not use SCons, followed by linking libmoo.a it produces with my project
Have the builds be perfectly parallel (scons -j9 for an 8-core?)
Have some debug/release-independent way of specifying libraries to link. Something like:
env.Program(target='foo', source=['foo/main.cpp', '#build/(DEBUG_OR_RELEASE)/lib/libsomething.a'])
What would the very basic SConstruct/SConscript files to do the above look like? Even just pointers in the right directions would be great too!
Thanks in advance :-)
I do this for builds for multiple platforms (rather than debug/release) but the concept's the same. The basic idea is that you need 2 files in the project root - a SConstruct to set up the build directories (or "variant directories" as they are known in scons), then a SConscript that describes the actual build steps.
In the SConstruct file you'd specify the variant directory and its corresponding source directory:
SConscript(dirs='.',
variant_dir=variant_dir,
duplicate=False,
exports="env")
Now you want variant_dir to depend on a flag. You'd use AddOption or Variables to do this. Here's one example of a complete top-level SConstruct to do that:
# build with `scons --debug-build` for debug.
AddOption(
'--debug-build',
action='store_true',
help='debug build',
default=False)
env = Environment()
if GetOption('debug_build'):
env.ParseFlags('-DDEBUG')
variant_dir = 'build/debug'
else:
variant_dir = 'build/release'
SConscript(dirs='.',
variant_dir=variant_dir,
duplicate=False,
exports="env")
AddOption is easiest to use, but if you use Variables then you can cache the result between runs, rather than having to spell out "scons --debug-build" each time.
All the directory setup and associated cruft is in the SConstruct. Now the SConscript file is quite simple and doesn't need to worry about build directories at all.
Import('env')
env.Program(target='foo_prog', source=['foo/main.cpp', 'lib/libmoo.a'])
# foo_prog since foo already exists as the name of the directory...
This is about the simplest way I've found to set up different build directories without getting weird errors. It's also pretty flexible - you can add different platform builds just by modifying the "env" in the top-level script without having to alter the actual meat of the build.
The only spanner in the works in your question is the way to compile autoconf-style projects directly from SCons. The easiest way is probably with a couple of Command() calls, but SCons likes to know about the inputs and outputs of each step, so this can get hacky. Also, you have to rely on the autoconf build having a correct VPATH setup - some projects don't work if you try and compile outside the source tree. Anyway, a way to compile autoconf projects would be something like this:
import os
Import('env')
# get the path to the configure script from the "moo" source directory
conf = env.File('moo/configure').srcnode().abspath
# Create the "moo" build directory in the build dir
build_dir = env.Dir('.').path
moo_dir = os.path.join(build_dir, 'moo')
Mkdir(moo_dir)
# run configure from within the moo dir
env.Command('moo/Makefile', 'moo/Makefile.am',
conf, chdir=moo_dir)
# run make in the moo dir
env.Command('moo/libmoo.a', 'moo/Makefile',
'make', chdir=moo_dir)
env.Program(target='foo_prog', source=['foo/main.cpp', 'moo/libmoo.a'])
Running the configure step from the source directory while the current working directory is somewhere in the build hierarchy is awkward. The make step is less messy, but still needs to know about the current build directory. Since you specify "libmoo.a" as an output of the make step and libmoo.a as an input to the program, all the dependencies Just Work, so a parallel build works fine. Parallel builds only break down when you fudge dependencies too much.
I know this is an old question, I just want to add an alternative to:
be able to know the current variant in the sconscript file (not only in the parent)
and to be able to build multiple variants in a single scons command
In the sconstruct file (the parent), we define a ListVariable named variants with the list of the variants that we allow (eg. ['release', 'debug']).
Then to be able to know the current variant in the sconscript file, we just loop option we have defined and export it into the sconscript.
I use genv as variable name to notate global environment:
# sconstruct
opts = Variables()
opts.AddVariables(
ListVariable('variants', 'list of variants to build', 'all', names = ['debug','release']),
)
genv = Environment( options = opts )
for variant in genv['variants']:
SConscript('sconscript', exports=['genv', 'variant'], variant_dir='#build/'+variant, duplicate=False)
In the sconscript file we Clone de genv and we can use the variant variable to do our setup in the local environment env:
# sconscript
Import('*')
import os.path
env = genv.Clone()
if variant == 'debug':
env.Append( CPPFLAGS = ['/Zi'])
src = 'src/hello.cpp'
app,ext = os.path.splitext(os.path.basename(src))
obj = env.Object ('obj/'+app, src)
bin = env.Program('bin/'+app, obj)
Using a ListVariable allows us to call
scons variants=release
or
scons variants=debug
or
scons variants=all
This last command (and the default command) builds all the variants.
There's a good solution to define multiple build modes ('debug', 'release') in the SCons Wiki:
http://www.scons.org/wiki/SconstructMultiple
That's how the richq SConstruct file would look like:
#get the mode flag from the command line
#default to 'release' if the user didn't specify
mymode = ARGUMENTS.get('mode', 'release')
#check if the user has been naughty: only 'debug' or 'release' allowed
if not (mymode in ['debug', 'release']):
print "Error: expected 'debug' or 'release', found: " + mymode
Exit(1)
#tell the user what we're doing
print '**** Compiling in ' + mymode + ' mode...'
env = Environment()
if mode == 'debug':
env.Append(CPPDEFINES = ['DEBUG'])
variant_dir = 'build/debug'
else:
variant_dir = 'build/release'
SConscript(dirs = '.', variant_dir = variant_dir, duplicate = False, exports = "env")
You then call scons mode=release (or just scons as the release is the default mode), or scons mode=debug.
Suppose I have an SConstruct file that looks like this:
env = Environment()
env.Program("a", ["a.c", "util.c"])
env.Program("b", ["b.c", "util.c"])
This build works properly with no SCons warning messages. However, if I modify this to specify different libraries for each Program build (the actual libraries are not relevant):
env.Program("a", ["a.c", "util.c"], LIBS="m")
env.Program("b", ["b.c", "util.c"], LIBS="c")
then I get the warning:
scons: warning: Two different environments were specified for target util.o,
but they appear to have the same action: $CC -o $TARGET -c $CFLAGS $CCFLAGS $_CCCOMCOM $SOURCES
This appears to be caused by the Program builder automatically creating a new environment for building the sources, even though it is just the LIBS variable that is different (and so only the link step needs to have a different environment). I can work around this by doing something like:
util = env.Object("util.c")
env.Program("a", ["a.c"] + util, LIBS="m")
env.Program("b", ["b.c"] + util, LIBS="c")
This uses a single Object builder for building util.c, then using the precompiled object file in each Program build, thus avoiding the warning. However, this should not really be necessary. Is there a more elegant way to work around this problem? Or is this actually a bug in SCons that should be fixed?
Context: I have nearly 2000 C source files compiled into about 20 libraries and 120 executables with lots of shared sources. I created the SConstruct file from the previous proprietary build system using a conversion script I wrote. There are about 450 "Two different environments" warning messages produced by SCons for a full build using my current SConstruct.
I found a workaround that doesn't involve creating extra variables to hold the object file nodes:
env.Program("a", ["a.c", env.Object("util.c")], LIBS="m")
env.Program("b", ["b.c", env.Object("util.c")], LIBS="c")
This isolates the build of util.c within a single environment. Although it is specified twice, once for each Program, SCons doesn't warn about this because it's the same source built with the same env object. Of course SCons only compiles the source once in this case.
You may use the Split function and a custom helper to simplify the build process for large projects:
def create_objs(SRCS, path=""):
return [env.Object(path+src+".cpp") for src in SRCS]
prg1 = Split("file_1 file_2 file_N")
prg2 = Split("file_2 file_5 file_8")
env.Program("a", create_objs(prg1), LIBS="x")
env.Program("b", create_objs(prg2), LIBS="y")
The object files are created only once, and they can be used in multiple builds. Hope this helps...
One issue I found in my code was that I was not using the target object path correctly. Or in otherwords I had a variant dir directive, but instead of using BUILDPATH i ended up using my original source code path. This way Scons was finding the object generated in target BUILDPATH and source path.
Creating a static library out of the first set of files and linking the library to the next set of files (which have some files in common with the first set) to create a target works as well.
env.StaticLibrary ("a", ["a.c","util.c"], LIBS = "m")
env.Program ("b", ["b.c","util.c"], LIBS = ["c","a"])
Is there an easy way to install SCons as an egg using zc.buildout? My initial thought was that since it uses distutils I would be able to install it using zc.recipe.egg and thought that the egg would be set up in such a way that the SCons scripts go to ${buildout:directory}/bin and SCons gets added to the PYTHONPATH.
[buildout]
parts =
python
[python]
recipe = zc.recipe.egg
interpreter = mython
eggs =
SCons == 2.0.1
Unfortunately, I don't think SCons' setup.py is written in such a way that it works well with zc.recipe.egg. I end up with:
${buildout:directory}/eggs/scons-2.0.1-py2.6.egg/
${buildout:directory}/eggs/scons-2.0.1-py2.6.egg/scons-2.0.1/
${buildout:directory}/eggs/scons-2.0.1-py2.6.egg/scons-2.0.1/SCons/
${buildout:directory}/eggs/scons-2.0.1-py2.6.egg/EGG-INFO/scripts/
This would work if (2) were on PYTHONPATH and the contents of (4) were copied or linked to the bin directory.
Ideally, I would like to install this as an egg to make use of my global eggs-directory; I don't want to have several copies of the same build tool. Is it possible to accomplish this goal using existing recipes, or should I write my own recipe?
EDIT: I made a personal copy and modified SCons' setup.py arguments so that a scons entry point will be created. The only changes I made: remove 'cmdclass', add 'entry_points'.
arguments = {
'name' : "scons",
...
'scripts' : scripts,
'entry_points' : {
'console_scripts':
['scons = SCons.Script:main']
}
}
This kind of works. It gives me a scons script in bin/scons, but not sconsign, scons-time, or scons.bat. I put it in a tar in /var/www/eggs and serve it with apache.
[buildout]
parts =
python
[python]
recipe = zc.recipe.egg:scripts
interpreter = mython
eggs =
scons == 2.0.1
find-links =
http://localhost/eggs/
Not sure if I want to continue in this direction.
Oh my. I looked into the setup.py for SCons, and it is a bit of a mess. It even stubbornly only installs in /usr/local when installing on a Mac. It certainly is not suitable for use with setuptools (and thus zc.buildout).
You can use zc.recipe.command to run the setup.py script with ${buildout:executable} setup.py and configure it to install in a parts subdir, then use a separate part to symlink all the scripts into bin/:
[buildout]
parts = scons
[scons-download]
recipe = gocept.download
url = http://prdownloads.sourceforge.net/scons/scons-2.0.1.tar.gz
md5sum = beca648b894cdbf85383fffc79516d18
[scons-install]
recipe = plone.recipe.command
command = ${buildout:executable} ${scons-download:location}/setup.py install --prefix=${buildout:parts-directory}/scons-install
location = ${buildout:parts-directory}/scons-install
[scons]
recipe = cns.recipe.symlink
symlink =
scons
scons-time
sconsign
symlink_base = ${scons-install:location}/bin
symlink_target = ${buildout:bin-directory}
So, we ignore setuptools altogether, do not build an egg, but instead just download the tarball, run setup.py manually, then symlink the interesting parts.