I just started learning to use SCons, anticipating that it solves some of my issues with make. I'm creating a source hierarchy to understand the basics of SCons.
Let's start with this folder structure:
test/foo: contains main.cpp, main.h
test/bar: contains its own main.cpp, main.h
test/common: contains utils.cpp and utils.h used by both foo and bar
test/external/moo: the source to some external library, containing 'configure' which produces 'Makefile' (not using SCons), so SCons needs to invoke 'make' after 'configure'; I suspect this part might be tricky when build dirs are used
test/build/debug: build dir for debug
test/build/release: build dir for release
Here's what I'd like to do:
Have two types of builds: debug/release where the only difference is that debug specifies -DDEBUG to g++
Use build dirs so that no .o files are created in my source tree. Let's call these build dirs "build/debug" and "build/release"
Be able to invoke ./configure and make on another project that does not use SCons, followed by linking libmoo.a it produces with my project
Have the builds be perfectly parallel (scons -j9 for an 8-core?)
Have some debug/release-independent way of specifying libraries to link. Something like:
env.Program(target='foo', source=['foo/main.cpp', '#build/(DEBUG_OR_RELEASE)/lib/libsomething.a'])
What would the very basic SConstruct/SConscript files to do the above look like? Even just pointers in the right directions would be great too!
Thanks in advance :-)
I do this for builds for multiple platforms (rather than debug/release) but the concept's the same. The basic idea is that you need 2 files in the project root - a SConstruct to set up the build directories (or "variant directories" as they are known in scons), then a SConscript that describes the actual build steps.
In the SConstruct file you'd specify the variant directory and its corresponding source directory:
SConscript(dirs='.',
variant_dir=variant_dir,
duplicate=False,
exports="env")
Now you want variant_dir to depend on a flag. You'd use AddOption or Variables to do this. Here's one example of a complete top-level SConstruct to do that:
# build with `scons --debug-build` for debug.
AddOption(
'--debug-build',
action='store_true',
help='debug build',
default=False)
env = Environment()
if GetOption('debug_build'):
env.ParseFlags('-DDEBUG')
variant_dir = 'build/debug'
else:
variant_dir = 'build/release'
SConscript(dirs='.',
variant_dir=variant_dir,
duplicate=False,
exports="env")
AddOption is easiest to use, but if you use Variables then you can cache the result between runs, rather than having to spell out "scons --debug-build" each time.
All the directory setup and associated cruft is in the SConstruct. Now the SConscript file is quite simple and doesn't need to worry about build directories at all.
Import('env')
env.Program(target='foo_prog', source=['foo/main.cpp', 'lib/libmoo.a'])
# foo_prog since foo already exists as the name of the directory...
This is about the simplest way I've found to set up different build directories without getting weird errors. It's also pretty flexible - you can add different platform builds just by modifying the "env" in the top-level script without having to alter the actual meat of the build.
The only spanner in the works in your question is the way to compile autoconf-style projects directly from SCons. The easiest way is probably with a couple of Command() calls, but SCons likes to know about the inputs and outputs of each step, so this can get hacky. Also, you have to rely on the autoconf build having a correct VPATH setup - some projects don't work if you try and compile outside the source tree. Anyway, a way to compile autoconf projects would be something like this:
import os
Import('env')
# get the path to the configure script from the "moo" source directory
conf = env.File('moo/configure').srcnode().abspath
# Create the "moo" build directory in the build dir
build_dir = env.Dir('.').path
moo_dir = os.path.join(build_dir, 'moo')
Mkdir(moo_dir)
# run configure from within the moo dir
env.Command('moo/Makefile', 'moo/Makefile.am',
conf, chdir=moo_dir)
# run make in the moo dir
env.Command('moo/libmoo.a', 'moo/Makefile',
'make', chdir=moo_dir)
env.Program(target='foo_prog', source=['foo/main.cpp', 'moo/libmoo.a'])
Running the configure step from the source directory while the current working directory is somewhere in the build hierarchy is awkward. The make step is less messy, but still needs to know about the current build directory. Since you specify "libmoo.a" as an output of the make step and libmoo.a as an input to the program, all the dependencies Just Work, so a parallel build works fine. Parallel builds only break down when you fudge dependencies too much.
I know this is an old question, I just want to add an alternative to:
be able to know the current variant in the sconscript file (not only in the parent)
and to be able to build multiple variants in a single scons command
In the sconstruct file (the parent), we define a ListVariable named variants with the list of the variants that we allow (eg. ['release', 'debug']).
Then to be able to know the current variant in the sconscript file, we just loop option we have defined and export it into the sconscript.
I use genv as variable name to notate global environment:
# sconstruct
opts = Variables()
opts.AddVariables(
ListVariable('variants', 'list of variants to build', 'all', names = ['debug','release']),
)
genv = Environment( options = opts )
for variant in genv['variants']:
SConscript('sconscript', exports=['genv', 'variant'], variant_dir='#build/'+variant, duplicate=False)
In the sconscript file we Clone de genv and we can use the variant variable to do our setup in the local environment env:
# sconscript
Import('*')
import os.path
env = genv.Clone()
if variant == 'debug':
env.Append( CPPFLAGS = ['/Zi'])
src = 'src/hello.cpp'
app,ext = os.path.splitext(os.path.basename(src))
obj = env.Object ('obj/'+app, src)
bin = env.Program('bin/'+app, obj)
Using a ListVariable allows us to call
scons variants=release
or
scons variants=debug
or
scons variants=all
This last command (and the default command) builds all the variants.
There's a good solution to define multiple build modes ('debug', 'release') in the SCons Wiki:
http://www.scons.org/wiki/SconstructMultiple
That's how the richq SConstruct file would look like:
#get the mode flag from the command line
#default to 'release' if the user didn't specify
mymode = ARGUMENTS.get('mode', 'release')
#check if the user has been naughty: only 'debug' or 'release' allowed
if not (mymode in ['debug', 'release']):
print "Error: expected 'debug' or 'release', found: " + mymode
Exit(1)
#tell the user what we're doing
print '**** Compiling in ' + mymode + ' mode...'
env = Environment()
if mode == 'debug':
env.Append(CPPDEFINES = ['DEBUG'])
variant_dir = 'build/debug'
else:
variant_dir = 'build/release'
SConscript(dirs = '.', variant_dir = variant_dir, duplicate = False, exports = "env")
You then call scons mode=release (or just scons as the release is the default mode), or scons mode=debug.
Related
I need to do purify build using SCons. Normally in nmake we would do
Nmake install -purify_7.0.1.0-008
How to do the same in SCons? Do I need to add any change in SConstruct file?
You'll need to prefix SHCC,CC, SHCXX, CXX, SHLINK, LINK with purify, and likely add the path to your purify binary to env['ENV']['PATH']
You'll need to do this anywhere (in the main SConstruct, all SConscripts, and if you have any code in site_scons) which creates an Environment() to use.
If you create one Enviroment() and that gets exported to your SConscripts where it's then env.Clone()'d, then you'd only have to change it in the initial env=Environment()
Here's an example (not tested):
env=Environment() # this is your existing code
for v in ['SHCC','CC','SHCXX','CXX','LINK','SHLINK']:
orig_value = env.subst('$%s'%v)
env['v'] = "purify "+ orig_value
print("Changing $%s from :\n%s\nto:\n%s",v, orig_value, env['v'])
I'm working on adding googletest unit testing to a large project.
We have a top level SConstruct, which calls SConscripts for each of the git submodules, which may further call other SConstructs contained in each component.
One of these SConscripts will build the googletest framework, which will create the 'gtest.h' file among others. I need to include 'gtest.h' in my source file (test1.cpp) which is in a different repo that has it's own SConscript.
How can I require the gtest.h in test SConscript file before test1.cpp consumes it?
Imagine something like:
Top level SConstruct:
env.SConscript('{path to Gtest}/SConscript')
env.SConscript('{path to my unit test}/SConscript')
SConscript for building Gtest:
env2 = env.Clone()
def buildGtest(target, source, env):
#Assuming I have a CMake file that does this in the directory...
subprocess.run(['cmake', '../'], cwd='build/')
subprocess.run(['make'], cwd='build/')
env2.Command(['gtest.h', 'build/libgtest.a', 'build/libgtest_main.a'], [], buildGtest)
SConscript for my unit test:
env2 = env.Clone()
env2.Require('{path to gtest}/build/include/gtest.h') # This doesn't seem to work
env2.Append(CPPPATH='{path to gtest}/build/include')
env2.Object(target = 'test1.o', source = 'test1.cpp')
You need to ensure that your CPPPATH includes the location where googletest generates the header file.
This is not great. You should specify at least one source file. I'm guessing you have a CmakeList.txt and a bunch of source files you should list here, so it will rebuild if they change.
env2.Command(['gtest.h', 'build/libgtest.a', 'build/libgtest_main.a'], [], buildGtest)
Additionally I'd change your buildGtest to the following.
buildGtest = ["cd build; cmake ../','cd build; make']
And get rid of your function.
env2.Command(['gtest.h', 'build/libgtest.a', 'build/libgtest_main.a'], [], buildGtest)
Then SCons will scan your source file and know that it requires that file.
You can check the dependency tree that SCons is aware of by running as
scons --tree=prune
It may be quite verbose, but you'll see all the dependencies.
You should also be able to run as
scons --tree=prune {path to my unit test}/test.o
I have defined two libraries - one static, one shared - to be built in a "libraries" subdirectory with an SConscript file. This SConscript is then invoked from SConstruct in the parent directory, which dilligently builds both libraries.
D:\tony\libraries\SConscript:
# Define environmental/project constants
CPPPATH = ['../headers']
SOURCES = ['greeter.cxx']
# Inherit the parent environment and update values if necessary.
Import('env')
# Build targets using defined environment variables.
print "Building libraries"
env.StaticLibrary(target = 'lib_greeter.a', source = SOURCES, CPPPATH = CPPPATH)
env.SharedLibrary(target = 'greeter.dll', source = SOURCES, CPPPATH = CPPPATH)
D:\tony\SConstruct:
# Define environmental/project constants
TOOLS = ['gcc', 'g++', 'gnulink', 'ar']
PATH = ['C:/cygwin/bin']
CYGWIN = ['nodosfilewarning']
DECIDER = 'MD5-timestamp' # Use a combination of timestamps and checksums to decide if dependencies have changed.
# Initialize the Default Environment and update values.
env = DefaultEnvironment(tools=TOOLS)
env['ENV']['PATH'] = PATH
env['ENV']['CYGWIN'] = CYGWIN
env.Decider(DECIDER)
# Call subsidiary SConscript files with defined environment variables.
SConscript('libraries/SConscript', exports = 'env', duplicate = 0) # do not copy src files to build directory.
But how can I specify that I only wish to build one of the libraries (eg. greeter.dll) when invoking scons on the cmd line?
I had previously defined some custom command line options using AddOption to introduce some flow control, but somehow that didn't feel quite right.
Your question can be read one of two ways:
How can I select the target to build by naming it on the command line.
How can I arrange things such that greeter.dll is built by default if no target is named on the command line.
If your question is about the first case, you can just name the path to the generated file when you invoke SCons. Given your files, I would expect that to look something like scons libraries\greeter.dll.
If you are asking about the second case, you can use env.Default("greeter.dll") in your SConscript, and then when you invoke SCons with no arguments it will be built automatically.
I'm building code with multiple environments, outputting to multiple target directories. The natural way to manage this seems to be with variant directories. So I might want to build the same set of files multiple times with different options and different VariantDirs. So I want to be able to have multiple SConscript files in different locations, all referring back to the same source directory.
One option I've tried is to do this:
SConstruct
src/test.cpp
src/magic/SConscript
This is my SConstruct:
env = Environment()
SConscript('src/magic/SConscript',
variant_dir = 'build/src',
src_dir = 'src',
exports={'env':env},
duplicate=0)
and this is src/magic/SConscript:
Import('env')
source = 'test.cpp'
env.Object(source)
I get this output:
scons: *** [build/src/magic/test.o] Source `src/magic/test.cpp' not found, needed by target `build/src/magic/test.o'.
This looks like both the variant_dir and src_dir are not being respected by Object, since neither mention magic at all.
Have I misunderstood how variant_dir/src_dir are meant to work, and what is the best way to build the same set of files with different targets?
Your file/folder hierarchy doesn't fit the build specification in your SConstruct/SConscript files.
Note how file paths in SCons are usually relative to the location of the current SConscript, so:
source = 'test.cpp'
env.Object(source)
in src/magic/SConscript gets expanded to src/magic/test.cpp...which obviously doesn't exist. You could use ../test.cpp as filename, or move the SConscript one up from src/magic to the src folder directly.
Some further remarks:
1.) When you specify a path for the name of the SConscript file in the SConscript call:
SConscript('src/SConscript',
variant_dir = 'build',
exports={'env':env},
duplicate=0)
SCons will automatically derive the src_dir argument from the path of the first argument.
2.) Please check out the chapter 14 "Hierarchical Builds" in the UserGuide ( http://www.scons.org/doc/production/HTML/scons-user.html ).
Suppose I have an SConstruct file that looks like this:
env = Environment()
env.Program("a", ["a.c", "util.c"])
env.Program("b", ["b.c", "util.c"])
This build works properly with no SCons warning messages. However, if I modify this to specify different libraries for each Program build (the actual libraries are not relevant):
env.Program("a", ["a.c", "util.c"], LIBS="m")
env.Program("b", ["b.c", "util.c"], LIBS="c")
then I get the warning:
scons: warning: Two different environments were specified for target util.o,
but they appear to have the same action: $CC -o $TARGET -c $CFLAGS $CCFLAGS $_CCCOMCOM $SOURCES
This appears to be caused by the Program builder automatically creating a new environment for building the sources, even though it is just the LIBS variable that is different (and so only the link step needs to have a different environment). I can work around this by doing something like:
util = env.Object("util.c")
env.Program("a", ["a.c"] + util, LIBS="m")
env.Program("b", ["b.c"] + util, LIBS="c")
This uses a single Object builder for building util.c, then using the precompiled object file in each Program build, thus avoiding the warning. However, this should not really be necessary. Is there a more elegant way to work around this problem? Or is this actually a bug in SCons that should be fixed?
Context: I have nearly 2000 C source files compiled into about 20 libraries and 120 executables with lots of shared sources. I created the SConstruct file from the previous proprietary build system using a conversion script I wrote. There are about 450 "Two different environments" warning messages produced by SCons for a full build using my current SConstruct.
I found a workaround that doesn't involve creating extra variables to hold the object file nodes:
env.Program("a", ["a.c", env.Object("util.c")], LIBS="m")
env.Program("b", ["b.c", env.Object("util.c")], LIBS="c")
This isolates the build of util.c within a single environment. Although it is specified twice, once for each Program, SCons doesn't warn about this because it's the same source built with the same env object. Of course SCons only compiles the source once in this case.
You may use the Split function and a custom helper to simplify the build process for large projects:
def create_objs(SRCS, path=""):
return [env.Object(path+src+".cpp") for src in SRCS]
prg1 = Split("file_1 file_2 file_N")
prg2 = Split("file_2 file_5 file_8")
env.Program("a", create_objs(prg1), LIBS="x")
env.Program("b", create_objs(prg2), LIBS="y")
The object files are created only once, and they can be used in multiple builds. Hope this helps...
One issue I found in my code was that I was not using the target object path correctly. Or in otherwords I had a variant dir directive, but instead of using BUILDPATH i ended up using my original source code path. This way Scons was finding the object generated in target BUILDPATH and source path.
Creating a static library out of the first set of files and linking the library to the next set of files (which have some files in common with the first set) to create a target works as well.
env.StaticLibrary ("a", ["a.c","util.c"], LIBS = "m")
env.Program ("b", ["b.c","util.c"], LIBS = ["c","a"])