In scons, how can I inject a target to be built? - scons

I want to inject a "Cleanup" target which depends on a number of other targets finishing before it goes off and gzip's some log files. It's important that I not gzip early as this can cause some of the tools to fail.
How can I inject a cleanup target for Scons to execute?
e.g. I have targets foo and bar. I want to inject a new custom target called 'cleanup' that depends on foo and bar and runs after they're both done, without the user having to specify
% scons foo cleanup
I want them to type:
% scons foo
but have scons execute as though the user had typed
% scons foo cleanup
I've tried creating the cleanup target and appending to sys.argv, but it seems that scons has already processed sys.argv by the time it gets to my code so it doesn't process the 'cleanup' target that I manually append to sys.argv.

you shouldn't use _Add_Targets or undocumented features, you can just add your cleanup target to BUILD_TARGETS:
from SCons.Script import BUILD_TARGETS
BUILD_TARGETS.append('cleanup')
if you use this documented list of targets instead of undocumented functions, scons won't be confused when doing its bookkeeping. This comment block can be found in SCons/Script/__init__.py:
# BUILD_TARGETS can be modified in the SConscript files. If so, we
# want to treat the modified BUILD_TARGETS list as if they specified
# targets on the command line. To do that, though, we need to know if
# BUILD_TARGETS was modified through "official" APIs or by hand. We do
# this by updating two lists in parallel, the documented BUILD_TARGETS
# list, above, and this internal _build_plus_default targets list which
# should only have "official" API changes. Then Script/Main.py can
# compare these two afterwards to figure out if the user added their
# own targets to BUILD_TARGETS.
so I guess it is intended to change BUILD_TARGETS instead of calling internal helper functions

One way is to have the gzip tool depend on the output of the log files. For example, if we have this C file, 'hello.c':
#include <stdio.h>
int main()
{
printf("hello world\n");
return 0;
}
And this SConstruct file:
#!/usr/bin/python
env = Environment()
hello = env.Program('hello', 'hello.c')
env.Default(hello)
env.Append(BUILDERS={'CreateLog':
Builder(action='$SOURCE.abspath > $TARGET', suffix='.log')})
log = env.CreateLog('hello', hello)
zipped_log = env.Zip('logs.zip', log)
env.Alias('cleanup', zipped_log)
Then running "scons cleanup" will run the needed steps in the correct order:
gcc -o hello.o -c hello.c
gcc -o hello hello.o
./hello > hello.log
zip(["logs.zip"], ["hello.log"])
This is not quite what you specified, but the only difference between this example and your requirement is that "cleanup" is the step that actually creates the zip file, so that is the step that you have to run. Its dependencies (running the program that generates the log, creating that program) are automatically calculated. You can now add the alias "foo" as follows to get the desired output:
env.Alias('foo', zipped_log)

In version 1.1.0.d20081104 of SCons, you can use the private internal SCons method:
SCons.Script._Add_Targets( [ 'MY_INJECTED_TARGET' ] )
If the user types:
% scons foo bar
The above code snippet will cause SCons to behave as though the user had typed:
% scons foo bar MY_INJECTED_TARGET

Related

Get scons to distinguish an empty and a non-existing source while rebuilding

While building my program it is important to distinguish between files that don't exists and files that are empty.
However, it appears that scons treats them the same and neglect to rebuild a target when a source file changed from one of these states to the other one.
Step by step example:
Step 0:
SConstruct
foo = Command('foo', [], 'echo $TARGET is not created here!')
bar = Command('bar', foo, 'touch $TARGET ; test -f $SOURCE')
Default(bar)
Result:
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
echo foo is not created here!
foo is not created here!
touch bar ; test -f foo
scons: *** [bar] Error 1
scons: building terminated because of errors.
My interpretation:
The command for foo fails to create the file but it doesn't raise and error so the command for bar is run. It checks if foo exists and returns an error. Build fails (everything as expected so far).
Step 1:
SConstruct:
foo = Command('foo', [], 'touch $TARGET')
bar = Command('bar', foo, 'touch $TARGET ; test -f $SOURCE')
Default(bar)
Result:
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
touch foo
touch bar ; test -f foo
scons: done building targets.
My interpretation:
foo is rebuilt because it has changed. This time it creates an empty file. bar is rebuild because it failed before. It succeeds this time. The build is successful (still as expected).
Step 2:
SConstruct
foo = Command('foo', [], 'echo $TARGET is not created here!')
bar = Command('bar', foo, 'touch $TARGET ; test -f $SOURCE')
Default(bar)
Result:
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
echo foo is not created here!
foo is not created here!
scons: `bar' is up to date.
scons: done building targets.
My interpretation:
foo is rebuilt because it has changed again (was restores to the previous version). The file foo doesn't exist any longer because scons removes files before building them and the command fails to recreate it. bar is not rebuilt because scons doesn't seem to detect a change in the source file.
The build is successful while it shouldn't!
How can I force scons to rebuild bar in the last step?
The solution should scale well to "foo" commands that produce many files, list of which is generated programmatically in SConscript and cannot be hard-coded.
By the way, scons does now distinguish between emtpy and nonexistent, that's a fairly recent change (commit 3b7f8b4ce0, github.com/SCons/scons/pull/3833).
The default way SCons determines is file has hanged is to compare MD5 signatures of the previous and the current versions of the file.
The signature for a non-existent file is calculated from 0 bytes of data just like for an empty file so SCons doesn't see a difference between them. This is normally OK, especially that not creating the target file isn't an entirely legitimate use of SCons.
However, we can make it work by supplying different function that decides if files are different.
Such function in SCons is called a Decider.
There are three of them provided out of the box. The default one uses MD5. The second one uses timestamp. The third one uses MD5 but only if the timestamp is different.
In this case, timestamp could perhaps work because it is 0 for a non-existent file. However, it would generate false-positives when timestamp changes and the contents of the file do not.
Instead, we can supply our own decider which will do exactly what we want it to:
from os import path
env = DefaultEnvironment()
decider_env = env.Clone()
def decide_if_changed(dependency, target, prev_ni, repo_node=None):
csig = dependency.get_csig() # it has to be called every time or the value won't be in `prev_ni` for the next check
return not path.isfile(dependency.abspath) or not hasattr(prev_ni, 'csig') or prev_ni.csig != csig
decider_env.Decider(decide_if_changed)
foo = env.Command('foo', [], 'echo $TARGET is not created here!')
bar = decider_env.Command('bar', foo, 'touch $TARGET ; test -f $SOURCE')
Default(bar)
This custom decider is similar to the default implementation based on MD5 except it also reports a change if the file doesn't exist.
This should cover the problem described in the question.
The new decider is assigned to a clone of the default environment. This way we have control over which target uses it. In this case only bar uses the non-default decider.
If you're saying "how can I force SCons to do xyz?", then you're understanding of SCons is incomplete.
SCons will only build targets which are out of date.
Unless..
You use AlwaysBuild(target) see: https://scons.org/doc/production/HTML/scons-man.html#f-AlwaysBuild
It also seems like you never want foo to be removed before it's (re)built?
Then you should use Precious(target) see: https://scons.org/doc/production/HTML/scons-man.html#f-Precious
Also.. it's bad form to call a builder with an empty source.
How would SCons ever know if it's out of date?
For your example what causes foo to be (re)built?

When changing the comment of a .c file, scons still re-compile it?

It's said that scons uses MD5 signature as default decider to dertermine whether a source file needs re-compilation. E.g. I've got SConstruct as below:
Library('o.c')
And my o.c is:
$ cat o.c
/*commented*/
#include<stdio.h>
int f(){
printf("hello\n");
return 2;
}
Run scons and remove the comment line, run scons again. I expect that scons should not compile it again, but actually it's:
gcc -o o.o -c o.c
scons: done building targets.
If I change SConstruct file to add one line:
Decider('MD5').
Still same result.
My question is: how to make sure that for scons, when changing source file comments, they don't get re-built?
Thanks!
As you correctly stated, SCons uses the MD5 hashsum of a source file to decide whether it has changed or not (content-based), and a rebuild of the target seems to be required (since one of its dependencies changed).
By adding or changing a comment, the MD5 sum of the file changes...so the trigger fires.
If you don't like this behaviour, you can write and use your own Decider function which will omit comment changes to your likings. Please check section 6.1.4 "Writing Your Own Custom Decider Function" in the UserGuide to see how this can be done.

Cmake add command line argument to binary

I create a binary myBinary via cmake/CMakeLists.txt.
I would like to "include" default options on my binary.
In other words, I want my binary to be called with myBinary --option myopt even when I just run ./myBinary
How can I do that?
CMake does not have built-in support for you you want to do.
One solution is to do as #Youka said - change the source code of your program.
Another solution that I have used sometimes is to autogenerate a script that executes an executable:
# Create startup script
MACRO(GEN_START_SCRIPT binName)
# Generate content
SET(fileContent
"#!/bin/bash\n"
"\n"
"# This startup script is auto generated - do not modify!\n"
"\n"
"${binName} -a 23 -b 34 -c 976\n"
"\n"
)
# Write to file
SET(fileName ${CMAKE_CURRENT_BINARY_DIR}/${binName}.sh)
FILE(WRITE ${fileName} ${fileContent})
ENDMACRO()
Then call the macro after defining your executable:
ADD_EXECUTABLE(myBinary file1.c file.2)
GEN_START_SCRIPT(myBinary)
You can of course add other stuff to the script, like environment variables etc.
If you're in control of the sources and you want different default behavior... change the sources!
This is in no way a build system issue (CMake or otherwise).

Create custom ./configure command line arguments

I'm updating a project to use autotools, and to maintain backwards compatibility with previous versions, I would like the user to be able to run ./configure --foo=bar to set a build option.
Based on reading the docs, it looks like I could set up ./configure --enable-foo, ./configure --with-foo, or ./configure foo=bar without any problem, but I'm not seeing anything allowing the desired behavior (specifically having a double dash -- before the option).
Any suggestions?
There's no way I know of doing this in configure.ac. You'll have to patch configure. This can be done by running the patching script in a bootstrap.sh after running autoreconf. You'll have to add your option to the ac_option processing loop. The case for --x looks like a promising one to copy or replace to inject your new option, something like:
--foo=*)
my_foo=$ac_optarg ;;
There's also some code that strips out commandline args when configure sometimes needs to be re-invoked. It'll be up to you to determine whether --foo should be stripped or not. I think this is probably why they don't allow this in the first place.
If it were me, I'd try and lobby for AC_ARG_WITH (e.g. --with-foo=bar). It seems like a lot less work.
in order to do that you have to add to your configure.ac something like this:
# Enable debugging mode
AC_ARG_ENABLE(debug,
AC_HELP_STRING([--enable-debug],[Show a lot of extra information when running]),
AM_CPPFLAGS="$AM_CPPFLAGS -DDEBUG"
debug_messages=yes,
debug_messages=no)
AC_SUBST(AM_CPPFLAGS)
AC_SUBST(AM_CXXFLAGS)
echo -e "\n--------- build environment -----------
Debug Mode : $debug_messages"
That is just a simple example to add for example a --enable-debug, it will set the DEBUG constant on the config.h file.
then your have to code something like this:
#include "config.h"
#ifdef DEBUG
// do debug
#else
// no debug
#endif

Autoconf check for program and fail if not found

I'm creating a project and using GNU Autoconf tools to do the configuring and making. I've set up all my library checking and header file checking but can't seem to figure out how to check if an executable exists on the system and fail if it doesn't exist.
I've tried:
AC_CHECK_PROG(TEST,testprogram,testprogram,AC_MSG_ERROR(Cannot find testprogram.))
When I configure it runs and outputs:
Checking for testprogram... find: `testprogram. 15426 5 ': No such file or directory
but does not fail.
I found this to be the shortest approach.
AC_CHECK_PROG(FFMPEG_CHECK,ffmpeg,yes)
AS_IF([test x"$FFMPEG_CHECK" != x"yes"], [AC_MSG_ERROR([Please install ffmpeg before configuring.])])
Try this which is what I just lifted from a project of mine, it looks for something called quantlib-config in the path:
# borrowed from a check for gnome in GNU gretl: def. a check for quantlib-config
AC_DEFUN(AC_PROG_QUANTLIB, [AC_CHECK_PROG(QUANTLIB,quantlib-config,yes)])
AC_PROG_QUANTLIB
if test x"${QUANTLIB}" == x"yes" ; then
# use quantlib-config for QL settings
[.... more stuff omitted here ...]
else
AC_MSG_ERROR([Please install QuantLib before trying to build RQuantLib.])
fi
Similar to the above, but has the advantage of also being able to interact with automake by exporting the condition variable
AC_CHECK_PROG([ffmpeg],[ffmpeg],[yes],[no])
AM_CONDITIONAL([FOUND_FFMPEG], [test "x$ffmpeg" = xyes])
AM_COND_IF([FOUND_FFMPEG],,[AC_MSG_ERROR([required program 'ffmpeg' not found.])])
When using AC_CHECK_PROG, this is the most concise version that I've run across is:
AC_CHECK_PROG(BOGUS,[bogus],[bogus],[no])
test "$BOGUS" == "no" && AC_MSG_ERROR([Required program 'bogus' not found.])
When the program is missing, this output will be generated:
./configure
...cut...
checking for bogus... no
configure: error: Required program 'bogus' not found.
Or when coupled with the built-in autoconf program checks, use this instead:
AC_PROG_YACC
AC_PROG_LEX
test "$YACC" == ":" && AC_MSG_ERROR([Required program 'bison' not found.])
test "$LEX" == ":" && AC_MSG_ERROR([Required program 'flex' not found.])
Stumbled here while looking for this issue, I should note that if you want to have your program just looked in pathm a runtime test is enough:
if ! which programname >/dev/null ; then
AC_MSG_ERROR([Missing programname]
fi
This is not exactly a short approach, it's rather a general purporse approach (although when there are dozens of programs to check it might be also the shortest approach). It's taken from a project of mine (the prefix NA_ stands for “Not Autotools”).
A general purpose macro
dnl ***************************************************************************
dnl NA_REQ_PROGS(prog1, [descr1][, prog2, [descr2][, etc., [...]]])
dnl
dnl Checks whether one or more programs have been provided by the user or can
dnl be retrieved automatically. For each program `progx` an uppercase variable
dnl named `PROGX` containing the path where `progx` is located will be created.
dnl If a program is not reachable and the user has not provided any path for it
dnl an error will be generated. The program names given to this function will
dnl be advertised among the `influential environment variables` visible when
dnl launching `./configure --help`.
dnl ***************************************************************************
AC_DEFUN([NA_REQ_PROGS], [
m4_if([$#], [0], [], [
AC_ARG_VAR(m4_translit([$1], [a-z], [A-Z]), [$2])
AS_IF([test "x#S|#{]m4_translit([$1], [a-z], [A-Z])[}" = x], [
AC_PATH_PROG(m4_translit([$1], [a-z], [A-Z]), [$1])
AS_IF([test "x#S|#{]m4_translit([$1], [a-z], [A-Z])[}" = x], [
AC_MSG_ERROR([$1 utility not found])
])
])
m4_if(m4_eval([$# + 1 >> 1]), [1], [], [NA_REQ_PROGS(m4_shift2($*))])
])
])
Sample usage
NA_REQ_PROGS(
[find], [Unix find utility],
[xargs], [Unix xargs utility],
[customprogram], [Some custom program],
[etcetera], [Et cetera]
)
So that within Makefile.am you can do
$(XARGS)
or
$(CUSTOMPROGRAM)
and so on.
Features
It advertises the programs among the “influential environment variables” visible when the final user launches ./configure --help, so that an alternative path to the program can be provided
A bash variable named with the same name of the program, but upper case, containing the path where the program is located, is created
En error is thrown if any of the programs given have not been found and the user has not provided any alternative path for them
The macro can take infinite (couples of) arguments
When you should use it
When the programs to be tested are vital for compiling your project, so that the user must be able to provide an alternative path for them and an error must be thrown if at least one program is not available at all
When condition #1 applies to more than one single program, in which case there is no need to write a general purpose macro and you should just use your own customized code

Resources