Scons Explicit Dependency - scons

I have a simple .cpp file that depends on jsoncpp. As a part of my build process I want Scons to untar jsoncpp (if it isn't already) and build it (if it isn't already) before attempting to compile app.cpp since app.cpp depends on some .h files that are zipped up inside of jsoncpp.tar.gz.
This is what I've tried so far:
env = Environment()
env.Program('app', 'app.cpp')
env.Depends('app.cpp', 'jsoncpp')
def build_jsoncpp(target, source, env):
shutil.rmtree("jsoncpp", ignore_errors=True)
mytar = tarfile.open(str(source[0]))
mytar.extractall()
print("Extracted jsoncpp")
env.Command("jsoncpp", ['jsoncpp.tar.gz'], build_jsoncpp)
However, Scons never prints "Extracted jsoncpp"... it always attempts to compile app.cpp and then promptly fails.
If I were using make, I could simply do something like:
app: jsoncpp.tar.gz
# Build app
jsoncpp.tar.gz:
# Extract and build here
And the order would be guaranteed.

You should take a look at the UnTarBuilder, as a means to extract a tarfile and have all of the extracted files be properly inserted into the dependency tree. But the following will get what you have working.
You want to avoid explicit dependencies, if possible. One of the many joys of SCons is letting it take care of your dependencies for you. So just list the source file you are depending on as one of the targets of your untar command builder.
To test this I created a tar file called jsoncpp.tar.gz containing just one file, app.cpp, with the following contents.
#include <iostream>
int main() {
std::cout << "Hello World" << std::endl;
return 0;
}
And updated your SConstruct to the following.
import shutil
import tarfile
env = Environment()
env.Program('app', 'app.cpp')
def build_jsoncpp(target, source, env):
shutil.rmtree("jsoncpp", ignore_errors=True)
mytar = tarfile.open(str(source[0]))
mytar.extractall()
print("Extracted jsoncpp")
env.Command(["app.cpp"], ['jsoncpp.tar.gz'], build_jsoncpp)
Because you list the required source file you depend on as a target of your command builder, it will handle the dependencies for you.
And when you run, you will see the following.
>> scons --version
SCons by Steven Knight et al.:
script: v2.3.4, 2014/09/27 12:51:43, by garyo on lubuntu
engine: v2.3.4, 2014/09/27 12:51:43, by garyo on lubuntu
engine path: ['/usr/lib/scons/SCons']
Copyright (c) 2001 - 2014 The SCons Foundation
>> tree
.
├── jsoncpp.tar.gz
└── SConstruct
0 directories, 2 files
>> scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
build_jsoncpp(["app.cpp"], ["jsoncpp.tar.gz"])
Extracted jsoncpp
g++ -o app.o -c app.cpp
g++ -o app app.o
scons: done building targets.
>> tree
.
├── app
├── app.cpp
├── app.o
├── jsoncpp.tar.gz
└── SConstruct
0 directories, 5 files
>> ./app
Hello World
The reason why your code does not work is because you are listing jsoncpp as the target of your untar command builder. Which is not a file that compiling app.cpp will depend on, even if you list that action as an explicit dependency.
While this doesn't exactly answer your question, I hope it provides a solution to what you are trying to accomplish.

It might help if you aligned the name of your builder function with your argument to the Command() method. ;)

Related

How can I require a generated file in my SConscript?

I'm working on adding googletest unit testing to a large project.
We have a top level SConstruct, which calls SConscripts for each of the git submodules, which may further call other SConstructs contained in each component.
One of these SConscripts will build the googletest framework, which will create the 'gtest.h' file among others. I need to include 'gtest.h' in my source file (test1.cpp) which is in a different repo that has it's own SConscript.
How can I require the gtest.h in test SConscript file before test1.cpp consumes it?
Imagine something like:
Top level SConstruct:
env.SConscript('{path to Gtest}/SConscript')
env.SConscript('{path to my unit test}/SConscript')
SConscript for building Gtest:
env2 = env.Clone()
def buildGtest(target, source, env):
#Assuming I have a CMake file that does this in the directory...
subprocess.run(['cmake', '../'], cwd='build/')
subprocess.run(['make'], cwd='build/')
env2.Command(['gtest.h', 'build/libgtest.a', 'build/libgtest_main.a'], [], buildGtest)
SConscript for my unit test:
env2 = env.Clone()
env2.Require('{path to gtest}/build/include/gtest.h') # This doesn't seem to work
env2.Append(CPPPATH='{path to gtest}/build/include')
env2.Object(target = 'test1.o', source = 'test1.cpp')
You need to ensure that your CPPPATH includes the location where googletest generates the header file.
This is not great. You should specify at least one source file. I'm guessing you have a CmakeList.txt and a bunch of source files you should list here, so it will rebuild if they change.
env2.Command(['gtest.h', 'build/libgtest.a', 'build/libgtest_main.a'], [], buildGtest)
Additionally I'd change your buildGtest to the following.
buildGtest = ["cd build; cmake ../','cd build; make']
And get rid of your function.
env2.Command(['gtest.h', 'build/libgtest.a', 'build/libgtest_main.a'], [], buildGtest)
Then SCons will scan your source file and know that it requires that file.
You can check the dependency tree that SCons is aware of by running as
scons --tree=prune
It may be quite verbose, but you'll see all the dependencies.
You should also be able to run as
scons --tree=prune {path to my unit test}/test.o

How can scons build SConscript in different directory

The directory hierarchy is like below.
├── prog_1
│ ├── hello.c
│ └── SConscript
└── folder
└── SConstruct
I tried to use the SConstruct to build the hello.c under prog_1, the SConstruct is as below:
SConscript('../prog_1/SConscript')
And the SConscript under prog_1 is as below:
Program('hello.c')
Whatever the absolute path or relative path to hello.c, I have tried but the scons will not recognize the path, it just ignored the SConscript specified under prog_1 and finished the build process. Is there any solution to make scons to enter other directory like this(non-subdirectory of SConstruct)?
SCons stays in the top-level folder of your build project by default. It doesn't "dive" into the single subfolders like make would do for example. Instead, all commands get constructed such that the folder-relative paths of source and target files you specify in a SConscript, are expanded to be valid from the top-level SConstruct. The background for this is, that SCons tries to avoid "chdir"s as much as possible such that builds can run fully in parallel.
Also by default, SCons will only build the files in the top-level folder or below, see #2 of our most frequently-asked frequently asked questions. In your case this is the folder "folder", where the SConstruct resides. The targets "prog_1" and "hello" are "outside the build tree" in this moment.
You can either explicitly specify a target each time you're calling SCons:
scons ../prog_1
, or move the "prog_1" folder down into the "folder" directory.
Please also read up on hierarchical and variant builds in chapters 14 and 15 of our User Guide.
SConscript('../prog_1/SConscript')
Change the SConscript under prog_1 is as below:
Default(Program('hello.c'))
Assuming you run scons in directory "folder", SCons will only try to build targets in the current directory or a subdirectory by default.
If you wish to have targets outside (in sibling or other directory trees), you need to either indicate in their SConscripts (Or in the SConstruct) that those targets should be built by default or run scons as
scons ../prog_1

SCons reports a dependency cycle when source and target file have the same suffix

I am trying to get SCons to call uglify on a series of JavaScript files, for continuous integration reasons...
However even though the 'duplicate=False' flag is passed to 'env.SConscript' calls; to prevent scons from copying sources to the build output folder. It flags files with the same source and target extension as a dependency cycle. Take for example the following simplified project structure:
SConstruct
applications/SConscript
applications/frontend/SConscript
applications/frontend/scripts/app.js
-- SContruct --
UglifyBuilder = env.Builder(
action = 'uglify --source $SOURCE --output $TARGET',
suffix = '.js',
single_source = True)
env['BUILDERS']['Uglify'] = UglifyBuilder
env.SConscript('applictions/SConscript', 'env',
variant_dir=os.path.join('build', 'debug', 'applications'),
duplicate=False)
-- applications/SConscript --
env.SConscript('frontend/SConscript', 'env', duplicate=False)
-- applications/frontend/SConscript --
env.Uglify('scripts/app.js')
When you run SCons on this (clean or not) the following output is generated; Stating that there is a dependency cycle between the target files:
$ scons applications/frontend/
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: building associated VariantDir targets: build/debug/applications/frontend
scons: `applications/frontend' is up to date.
scons: done building targets.
scons: *** Found dependency cycle(s):
build/debug/applications/frontend/scripts/app.js -> build/debug/applications/frontend/scripts/app.js
I also tried to pass an absolute path to the source file, but unfortunately with the same result. And sure I could change the output suffix to '.min.js', but there are other cases to come up with that output files with the same leader-path/filename as some arbitrary input source.
Am I misunderstanding SCons here? Or can somebody point me in the right direction of solving this?
Because if this is one of those cases where 'it isn't a bug it's a feature' applies I am going to pull my hair out. :)
Thanks!
I'm really sorry for you and your hair, but this behaviour doesn't have anything to do with variant_dir, and the "duplicate" option won't help you out either.
What your example boils down to is:
You have a file "app.js".
You have an Uglify Builder, mapping *.js files to *.js files.
You tell this Builder to do his stuff with "app.js".
Of course this leads to a dependency cycle, because your source and target have the same filename. Note, that SCons detects the cycle from build/debug/.../app.js to itself...and not to applications/frontend/scripts/app.js.
For breaking this cycle, you have to either rename one of your suffixes (*.in for source files?), or the file stem, e.g. you could name your source file "app_pretty.js" and then call
env.Uglify('scripts/app.js', 'scripts/app_pretty.js')

SCons: Separate debug/release build dirs with a hierarchical build

I just started learning to use SCons, anticipating that it solves some of my issues with make. I'm creating a source hierarchy to understand the basics of SCons.
Let's start with this folder structure:
test/foo: contains main.cpp, main.h
test/bar: contains its own main.cpp, main.h
test/common: contains utils.cpp and utils.h used by both foo and bar
test/external/moo: the source to some external library, containing 'configure' which produces 'Makefile' (not using SCons), so SCons needs to invoke 'make' after 'configure'; I suspect this part might be tricky when build dirs are used
test/build/debug: build dir for debug
test/build/release: build dir for release
Here's what I'd like to do:
Have two types of builds: debug/release where the only difference is that debug specifies -DDEBUG to g++
Use build dirs so that no .o files are created in my source tree. Let's call these build dirs "build/debug" and "build/release"
Be able to invoke ./configure and make on another project that does not use SCons, followed by linking libmoo.a it produces with my project
Have the builds be perfectly parallel (scons -j9 for an 8-core?)
Have some debug/release-independent way of specifying libraries to link. Something like:
env.Program(target='foo', source=['foo/main.cpp', '#build/(DEBUG_OR_RELEASE)/lib/libsomething.a'])
What would the very basic SConstruct/SConscript files to do the above look like? Even just pointers in the right directions would be great too!
Thanks in advance :-)
I do this for builds for multiple platforms (rather than debug/release) but the concept's the same. The basic idea is that you need 2 files in the project root - a SConstruct to set up the build directories (or "variant directories" as they are known in scons), then a SConscript that describes the actual build steps.
In the SConstruct file you'd specify the variant directory and its corresponding source directory:
SConscript(dirs='.',
variant_dir=variant_dir,
duplicate=False,
exports="env")
Now you want variant_dir to depend on a flag. You'd use AddOption or Variables to do this. Here's one example of a complete top-level SConstruct to do that:
# build with `scons --debug-build` for debug.
AddOption(
'--debug-build',
action='store_true',
help='debug build',
default=False)
env = Environment()
if GetOption('debug_build'):
env.ParseFlags('-DDEBUG')
variant_dir = 'build/debug'
else:
variant_dir = 'build/release'
SConscript(dirs='.',
variant_dir=variant_dir,
duplicate=False,
exports="env")
AddOption is easiest to use, but if you use Variables then you can cache the result between runs, rather than having to spell out "scons --debug-build" each time.
All the directory setup and associated cruft is in the SConstruct. Now the SConscript file is quite simple and doesn't need to worry about build directories at all.
Import('env')
env.Program(target='foo_prog', source=['foo/main.cpp', 'lib/libmoo.a'])
# foo_prog since foo already exists as the name of the directory...
This is about the simplest way I've found to set up different build directories without getting weird errors. It's also pretty flexible - you can add different platform builds just by modifying the "env" in the top-level script without having to alter the actual meat of the build.
The only spanner in the works in your question is the way to compile autoconf-style projects directly from SCons. The easiest way is probably with a couple of Command() calls, but SCons likes to know about the inputs and outputs of each step, so this can get hacky. Also, you have to rely on the autoconf build having a correct VPATH setup - some projects don't work if you try and compile outside the source tree. Anyway, a way to compile autoconf projects would be something like this:
import os
Import('env')
# get the path to the configure script from the "moo" source directory
conf = env.File('moo/configure').srcnode().abspath
# Create the "moo" build directory in the build dir
build_dir = env.Dir('.').path
moo_dir = os.path.join(build_dir, 'moo')
Mkdir(moo_dir)
# run configure from within the moo dir
env.Command('moo/Makefile', 'moo/Makefile.am',
conf, chdir=moo_dir)
# run make in the moo dir
env.Command('moo/libmoo.a', 'moo/Makefile',
'make', chdir=moo_dir)
env.Program(target='foo_prog', source=['foo/main.cpp', 'moo/libmoo.a'])
Running the configure step from the source directory while the current working directory is somewhere in the build hierarchy is awkward. The make step is less messy, but still needs to know about the current build directory. Since you specify "libmoo.a" as an output of the make step and libmoo.a as an input to the program, all the dependencies Just Work, so a parallel build works fine. Parallel builds only break down when you fudge dependencies too much.
I know this is an old question, I just want to add an alternative to:
be able to know the current variant in the sconscript file (not only in the parent)
and to be able to build multiple variants in a single scons command
In the sconstruct file (the parent), we define a ListVariable named variants with the list of the variants that we allow (eg. ['release', 'debug']).
Then to be able to know the current variant in the sconscript file, we just loop option we have defined and export it into the sconscript.
I use genv as variable name to notate global environment:
# sconstruct
opts = Variables()
opts.AddVariables(
ListVariable('variants', 'list of variants to build', 'all', names = ['debug','release']),
)
genv = Environment( options = opts )
for variant in genv['variants']:
SConscript('sconscript', exports=['genv', 'variant'], variant_dir='#build/'+variant, duplicate=False)
In the sconscript file we Clone de genv and we can use the variant variable to do our setup in the local environment env:
# sconscript
Import('*')
import os.path
env = genv.Clone()
if variant == 'debug':
env.Append( CPPFLAGS = ['/Zi'])
src = 'src/hello.cpp'
app,ext = os.path.splitext(os.path.basename(src))
obj = env.Object ('obj/'+app, src)
bin = env.Program('bin/'+app, obj)
Using a ListVariable allows us to call
scons variants=release
or
scons variants=debug
or
scons variants=all
This last command (and the default command) builds all the variants.
There's a good solution to define multiple build modes ('debug', 'release') in the SCons Wiki:
http://www.scons.org/wiki/SconstructMultiple
That's how the richq SConstruct file would look like:
#get the mode flag from the command line
#default to 'release' if the user didn't specify
mymode = ARGUMENTS.get('mode', 'release')
#check if the user has been naughty: only 'debug' or 'release' allowed
if not (mymode in ['debug', 'release']):
print "Error: expected 'debug' or 'release', found: " + mymode
Exit(1)
#tell the user what we're doing
print '**** Compiling in ' + mymode + ' mode...'
env = Environment()
if mode == 'debug':
env.Append(CPPDEFINES = ['DEBUG'])
variant_dir = 'build/debug'
else:
variant_dir = 'build/release'
SConscript(dirs = '.', variant_dir = variant_dir, duplicate = False, exports = "env")
You then call scons mode=release (or just scons as the release is the default mode), or scons mode=debug.

scons: making glob play nice with a build directory

I would like to build all .c files in a subdirectory. I figured I would do something like this:
src/foo/SConscript contains:
import glob;
here = Dir('.');
sourcefiles_raw = glob.glob(here.path+'/*.c');
print(sourcefiles_raw);
# print them for debugging
# ... then build them (in the process, making scons aware of dependencies)
src/SConscript contains:
SConscript(['foo/SConscript']);
SConstruct contains:
SConscript(['src/SConscript'],build_dir='build');
But it prints [], since glob.glob() runs in the directory build/foo before scons can decide which sourcefiles need to be copied from src/foo to build/foo.
How can I solve this problem?
never mind, it seems you're supposed to (RTFM) and use scons's Glob() rather than glob.glob().

Resources