I can use the Install() builder to install a directory, provided I don't use a variant.
Example without a variant (works fine):
$ mkdir -p test/src/doc
$ cd test
$ echo "SConscript('src/SConscript', exports={'dst': '/tmp'})" > SConstruct
$ echo xyz > src/doc/file
$ echo "Import('dst')" > src/SConscript
$ echo "Install(dst, 'doc')" >> src/SConscript
$ scons /tmp
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
Install directory: "src/doc" as "/tmp/doc"
scons: done building targets.
$ scons --version
SCons by Steven Knight et al.:
script: v2.1.0.r5357[MODIFIED], 2011/09/09 21:31:03, by bdeegan on ubuntu
engine: v2.1.0.r5357[MODIFIED], 2011/09/09 21:31:03, by bdeegan on ubuntu
engine path: ['/usr/lib/scons/SCons']
Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011 The SCons Foundation
Example with a variant (does not work):
$ echo "SConscript('src/SConscript', variant_dir='build', duplicate=0, exports={'dst': 'install'})" > SConstruct
$ scons build
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
Install file: "build/doc" as "build/install/doc"
scons: *** [build/install/doc] build/doc: No such file or directory
scons: building terminated because of errors.
This appears to be a very odd behaviour... For my current project, I want to be able to install a directory tree (typically documentation), but using variants.
Could anyone help me? Many thanks in advance!
After investigation, it seems scons tries to install directories using the given path relative from the variant directory, which obviously points to a non-existent folder.
I solved the problem by using absolute paths. Using my toy example:
$ mkdir -p test/src/doc
$ cd test
$ echo "import os" > SConstruct
$ echo "SConscript('src/SConscript', variant_dir='build', duplicate=0, exports={'cwd': os.getcwd() + '/src', 'dst': '/tmp'})" >> SConstruct
$ echo xyz > src/doc/file
$ echo "Import('cwd dst')" > SConscript
$ echo "Install(dst, cwd + '/doc')" >> SConscript
$ scons /tmp
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
Install directory: "src/doc" as "/tmp/doc"
scons: done building targets.
Problem solved! Hope this helps.
Related
While building my program it is important to distinguish between files that don't exists and files that are empty.
However, it appears that scons treats them the same and neglect to rebuild a target when a source file changed from one of these states to the other one.
Step by step example:
Step 0:
SConstruct
foo = Command('foo', [], 'echo $TARGET is not created here!')
bar = Command('bar', foo, 'touch $TARGET ; test -f $SOURCE')
Default(bar)
Result:
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
echo foo is not created here!
foo is not created here!
touch bar ; test -f foo
scons: *** [bar] Error 1
scons: building terminated because of errors.
My interpretation:
The command for foo fails to create the file but it doesn't raise and error so the command for bar is run. It checks if foo exists and returns an error. Build fails (everything as expected so far).
Step 1:
SConstruct:
foo = Command('foo', [], 'touch $TARGET')
bar = Command('bar', foo, 'touch $TARGET ; test -f $SOURCE')
Default(bar)
Result:
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
touch foo
touch bar ; test -f foo
scons: done building targets.
My interpretation:
foo is rebuilt because it has changed. This time it creates an empty file. bar is rebuild because it failed before. It succeeds this time. The build is successful (still as expected).
Step 2:
SConstruct
foo = Command('foo', [], 'echo $TARGET is not created here!')
bar = Command('bar', foo, 'touch $TARGET ; test -f $SOURCE')
Default(bar)
Result:
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
echo foo is not created here!
foo is not created here!
scons: `bar' is up to date.
scons: done building targets.
My interpretation:
foo is rebuilt because it has changed again (was restores to the previous version). The file foo doesn't exist any longer because scons removes files before building them and the command fails to recreate it. bar is not rebuilt because scons doesn't seem to detect a change in the source file.
The build is successful while it shouldn't!
How can I force scons to rebuild bar in the last step?
The solution should scale well to "foo" commands that produce many files, list of which is generated programmatically in SConscript and cannot be hard-coded.
By the way, scons does now distinguish between emtpy and nonexistent, that's a fairly recent change (commit 3b7f8b4ce0, github.com/SCons/scons/pull/3833).
The default way SCons determines is file has hanged is to compare MD5 signatures of the previous and the current versions of the file.
The signature for a non-existent file is calculated from 0 bytes of data just like for an empty file so SCons doesn't see a difference between them. This is normally OK, especially that not creating the target file isn't an entirely legitimate use of SCons.
However, we can make it work by supplying different function that decides if files are different.
Such function in SCons is called a Decider.
There are three of them provided out of the box. The default one uses MD5. The second one uses timestamp. The third one uses MD5 but only if the timestamp is different.
In this case, timestamp could perhaps work because it is 0 for a non-existent file. However, it would generate false-positives when timestamp changes and the contents of the file do not.
Instead, we can supply our own decider which will do exactly what we want it to:
from os import path
env = DefaultEnvironment()
decider_env = env.Clone()
def decide_if_changed(dependency, target, prev_ni, repo_node=None):
csig = dependency.get_csig() # it has to be called every time or the value won't be in `prev_ni` for the next check
return not path.isfile(dependency.abspath) or not hasattr(prev_ni, 'csig') or prev_ni.csig != csig
decider_env.Decider(decide_if_changed)
foo = env.Command('foo', [], 'echo $TARGET is not created here!')
bar = decider_env.Command('bar', foo, 'touch $TARGET ; test -f $SOURCE')
Default(bar)
This custom decider is similar to the default implementation based on MD5 except it also reports a change if the file doesn't exist.
This should cover the problem described in the question.
The new decider is assigned to a clone of the default environment. This way we have control over which target uses it. In this case only bar uses the non-default decider.
If you're saying "how can I force SCons to do xyz?", then you're understanding of SCons is incomplete.
SCons will only build targets which are out of date.
Unless..
You use AlwaysBuild(target) see: https://scons.org/doc/production/HTML/scons-man.html#f-AlwaysBuild
It also seems like you never want foo to be removed before it's (re)built?
Then you should use Precious(target) see: https://scons.org/doc/production/HTML/scons-man.html#f-Precious
Also.. it's bad form to call a builder with an empty source.
How would SCons ever know if it's out of date?
For your example what causes foo to be (re)built?
In a SCons build setup that uses quite some custom build actions, I had a lengthy action that was repeatedly retriggered because a target name was misspelled. Is it possible to configure SCons such that it the targets of a builder (or all builders) is really generated to prevent such cases?
For example, given
target = Command('some_target_file',
'some_input',
'echo foo > wrong_target_file'
)
with > scons --debug=explain will always result in
scons: building `some_target_file' because it doesn't exist
echo foo > wrong_target_file
without failure. While I would like to get an error in the spirit of
Error: target 'some_target_file' was not generated
I could emulate the desired behaviour using something like
dummy = Command('dummy', 'some_target_file', 'cat $SOURCE')
Default ([target, dummy])
resulting in
...
cat some_target_file
The system cannot find the file specified.
scons: *** [dummy] Error 1
scons: building terminated because of errors.
I'm new to Scons and I'm trying to figure out if I could use it for my use-case. I have a script whose main actoin is to take a single input and produces multiple output files in a given directory. However, it also needs one additional input and one additional output, as in
script --special-in some.foo --special-in some.bar input.foo output.dir/
The names of some.* files can be computed from the input file name (here input.foo). And the some.* files produced by one rule are consumed by other rules.
In the documentation I found that one can create custom builders as in
bld = Builder(action = 'foobuild $TARGETS - $SOURCES',
suffix = '.foo',
src_suffix = '.input',
emitter = modify_targets)
where the emitter adds the additional target and source. However, I couldn't find how should I distinguish the main source/target from the special ones, which need to be passed using specific options - I can't use $TARGETS and $SOURCES as in the above example. I could probably use a generator and index into source and target, but this seems a bit hacky. I there a better way?
From what you describe, you should be using both an emitter and a generator, just as you state at the end of your question. The "main" source/target will be the first element in the source/target lists. This doesn't seem hacky to me, but I may just be used to it...
Answers are always better with a working example...
Here is the SConstruct to do what you describe. I'm not exactly sure how you plan to compute some.foo and some.bar from input.foo, so in this example I compute input.bar and input.baz from input.foo, and just append output.dir to the list of targets.
import os
def my_generator(source, target, env, for_signature):
command = './script '
command += ' '.join(['--special-in %s' % str(i) for i in source[1:]])
command += ' '
command += ' '.join([str(t) for t in target])
return command
def my_emitter(target, source, env):
source += ['%s%s' % (os.path.splitext(
str(source[0]))[0], ext) for ext in ['.bar', '.baz']]
target += ['output.dir']
return target, source
bld = Builder(generator=my_generator,
emitter=my_emitter)
env = Environment(BUILDERS={'Foo':bld})
env.Foo('output.foo', 'input.foo')
When run on linux...
>> touch input.bar input.baz input.foo
>> echo "#\!/bin/sh" > script && chmod +x script
>> tree
.
├── input.bar
├── input.baz
├── input.foo
├── SConstruct
└── script
0 directories, 5 files
>> scons --version
SCons by Steven Knight et al.:
script: v2.3.4, 2014/09/27 12:51:43, by garyo on lubuntu
engine: v2.3.4, 2014/09/27 12:51:43, by garyo on lubuntu
engine path: ['/usr/lib/scons/SCons']
Copyright (c) 2001 - 2014 The SCons Foundation
>> scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
./script --special-in input.bar --special-in input.baz output.foo output.dir
scons: done building targets.
All dependencies/targets will be maintained, if you need to feed the outputs from one builder like this into another.
If this doesn't answer your question, please clarify what more you are trying to do.
How can I use AS_INIT_GENERATE to generate a script that is not in the same directory as the configure script, in particular so that VPATH builds will be honoured?
For example, for a configure.ac file containing
AC_PREREQ([2.68])
AC_INIT([example],[0.1])
AS_INIT_GENERATED([src/file.sh]) || AS_EXIT
AC_OUTPUT
running the commands
~ $ autoreconf .
~ $ mkdir build && cd build
~/build $ ../configure
results in the error message
../configure: line 1648: src/file.sh: No such file or directory
../configure: line 1655: src/file.sh: No such file or directory
I guess I'd have to make sure that the src directory exists before I call AS_INIT_GENERATE to create src/file.sh, or maybe I'm doing it all wrong?
Try something like this:
AC_PREREQ([2.68])
AC_INIT([example],[0.1])
test -d src || AS_MKDIR_P([src]) dnl <----- Create 'src' if it doesn't exist.
AS_INIT_GENERATED([src/file.sh]) || AS_EXIT
AC_OUTPUT
I'm trying to get SCons to copy a Makefile project from the source dir to the build dir and run some commands to produce libmoo.a, but I'm running into a dependency cycle error. Details follow:
./SConstruct:
env = Environment()
Export('env')
dirs = ['.']
variant_dir = 'build'
for dir in dirs:
SConscript(dir + '/' + 'SConscript', variant_dir=variant_dir + '/' + dir, duplicate=0)
./SConscript:
import os
Import('env')
env.Command(env.Dir('moo2').abspath, env.Dir('#moo').abspath, ["echo copying moo to the build dir", Copy("$TARGET", "$SOURCE")])
env.Command(env.Dir('moo2/Makefile').abspath, env.Dir('moo2').abspath, 'echo would run moo2.configure')
moolib = env.Command(env.Dir('moo2/libmoo.a').abspath, env.Dir('moo2/Makefile').abspath, 'echo would run make')
Default(moolib)
Error running scons:
scons: *** Found dependency cycle(s):
build/moo2/Makefile -> build/moo2 -> build/moo2/Makefile
build/moo2/libmoo.a -> build/moo2 -> build/moo2/Makefile -> build/moo2/libmoo.a
Also tried without using .abspath, but that shouldn't matter, right?
I don't see any cycles:
build/moo2/libmoo.a requires build/moo2/Makefile
build/moo2/Makefile requires build/moo2
build/moo2 requires (source/)moo
How is scons seeing a cycle? It seems to think that build/moo2/Makefile depends on build/moo2/libmoo.a, which is not what I intended to specify.
Any related suggestions are also welcome :-)
There's no need for the env.Dir('moo').abspath anywhere in your SConscript. So that would change it to:
Import('env')
env.Command('moo2', '#moo', ["echo copying moo to the build dir", Copy("$TARGET", "$SOURCE")])
env.Command('moo2/Makefile', 'moo2', 'echo would run moo2.configure') # Look Here
moolib = env.Command('moo2/libmoo.a', 'moo2/Makefile', 'echo would run make')
Default(moolib)
But that still yields the same error:
scons: *** Found dependency cycle(s):
build/moo2/Makefile -> build/moo2 -> build/moo2/Makefile
build/moo2/libmoo.a -> build/moo2/Makefile -> build/moo2/libmoo.a
So why is that?
SCons automatically makes a directory depend on all the files contained within.
See the line with the "# Look Here" comment. You've added a dependency moo2/Makefile now depends on moo2. moo2 depends on all it's contents by default, and thus your cycle.
So how do we fix it?
Import('env')
env.Install('moo2',Glob('#moo/*'))
env.Command('moo2/Makefile', env.Glob('moo2/*'), 'echo would run moo2.configure')
moolib = env.Command('moo2/libmoo.a', 'moo2/Makefile', 'echo would run make')
Default(moolib)
I've changed from using Copy() to env.Install(). Since Copy is not attached to the build Environment() object, it won't know about the variant dir. Install() and Copy() are functionally equivalent, with env.Install() being env aware.
Additionally I have it copying/depending on all the files in the directory, rather than the directory itself.
Now let's give that a try:
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python /Users/bdbaddog/devel/scons/trunk/bootstrap/src/script/scons.py --tree=prune
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
Install file: "moo/abc.c" as "build/moo2/abc.c"
Install file: "moo/configure" as "build/moo2/configure"
Install file: "moo/def.c" as "build/moo2/def.c"
echo would run moo2.configure
would run moo2.configure
echo would run make
would run make
+-build/moo2/libmoo.a
+-build/moo2/Makefile
| +-build/moo2/abc.c
| | +-moo/abc.c
| +-build/moo2/configure
| | +-moo/configure
| +-build/moo2/def.c
| | +-moo/def.c
| +-/bin/echo
+-/bin/echo
scons: done building targets.
Note the "--tree=prune" this flag will have SCons print out the dependency tree and prune duplications in the tree (so if a 2 files depend on the same tree of files, you'll only see it once)