I want to be able to call avrdude from SCons as a target. For example, running scons erase-device should run the avrdude command for doing so.
I'm attempting to do this by creating Builder objects that call avrdude and adding them to the environment.
# a string forming a base avrdude command that we can just add on to in the targets
avrdude_base = 'avrdude -p ' + env['MCU'] + ' -c ' + icspdevice
# target to erase everything--flash, EEPROM, and lock bits (but not fuse bits)
erase_dev = Builder(action = avrdude_base + ' -e')
env.Append(BUILDERS = {'EraseDevice' : erase_dev})
ed = env.EraseDevice()
eda = env.Alias('erase-device', ed)
env.AlwaysBuild(eda)
# target to write the AVR fuses and lock bits
write_fuse = Builder(action = avrdude_base + ' -U lfuse:w:' + lfuse + ':m -U hfuse:w:' + hfuse +
':m -U efuse:w:' + efuse + ':m -U lock:w:' + lockbits + ':m')
env.Append(BUILDERS = {'WriteFuses' : write_fuse})
wf = env.WriteFuses()
wfa = env.Alias('write-fuses', wf)
env.AlwaysBuild(wfa)
With this code, scons always exits saying that there is nothing to do. I think this is because, the way the code is shown, I don't give any source files to these Builders (env.EraseDevice() and env.WriteFuses()); therefore, SCons assumes that they don't need to be called.
So that's what I tried next. I just passed in an existing filename into those two Builders to make scons happy, even though it isn't needed. The problem now is that, regardless of whether I want to run scons write-fuses, scons erase-flash, or other targets that use avrdude, scons acts as though I'm trying to write the fuses. If, for example, the file name I passed in were foo.hex, then scons now thinks that it has to run the write-fuses target every time because scons thinks that 'avrdude' was supposed to generate an output file called foo, but that file is never generated.
Also, doing this means that I have to build the hex file before erasing the device or programming the fuse bits, which wouldn't normally be necessary.
How can I create targets in SCons that do not require any sources for input and that do not generate any output?
Thanks!
You're on the right track in saying that scons isn't running anything because it doesn't have any sources to turn into outputs, but the key is that scons wants to generate targets and it didn't think there were any to build.
One easy workaround is to give the erase-device and write-fuse commands dummy targets. These target files will never be generated, so if scons determines that this target needs to be built (because it was specified on the command line or is a dependency of something on the command line), scons will always run the appropriate avrdude ... command.
I think the use of Builders is adding extra complexity that you don't need. Builders are good for creating new source to target mappings, but you don't really need to involve files.
ed = env.Command('erase.dummy', [], avrdude_base + ' -e')
ed = env.EraseDevice()
env.AlwaysBuild(ed)
env.Alias('erase-device', ed)
...
As a side note, scons --tree=all is a nice way to see scons' calculated dependency tree. If you're mystified by what scons is doing, seeing the dependency tree can help with debug where your model diverges from scons.
Related
Like
make,
scons has a large number of predefined variables and rules. (Try scons | wc on an SConstruct containing env = Environment(); print(env.Dump()) to see how extended the set is.)
But suppose we aren't after the wizardry of presets but rather want to do something a lot more primitive—simulating launching a few instructions from the (bash, etc) command line?
Also suppose we're quite happy with the default Decider('MD5'). What is the translation of the one-souce-one-target:
out/turquoise.xyz: out/chartreuse.xyz
chartreuse_to_turquoise $< $#
of the two-source-one-target:
out/purple.xyz: out/lilac.xyz out/salmon.xyz
gen_purple $< $#
and of:
run_this:
python prog.py
which we would run on-demand by typing make run_this?
What does the SConstruct for these elementary constructs look like?
All the answers you're looking for are in the users guide (and manpage)
Firstly, assuming you don't want to scan the input files to add included files specified in the input files, you can use Commmand()
(See info here: https://scons.org/doc/production/HTML/scons-user.html#chap-builders-commands)
Then you'll want an alias to specify an a non file command line target
(See here:https://scons.org/doc/production/HTML/scons-user.html#chap-alias)
Putting those two together yields
env=Environment()
# one source, one target
env.Command('out/turquoise.xyz', 'out/chartreuse.xyz', 'chartreuse_to_turquoise $SOURCE $TARGET')
# Two source, one target
env.Command('out/purple.xyz',['out/lilac.xyz','out/salmon.xyz'], 'gen_purple $SOURCES $TARGET')
# And your .phony make target which is actually not great for reproducibility and determining when it should be rerun, because you do not specify any sources or targets
env.Alias('run_this','python prog.py')
Note: SCons doesn't NOT propagate your shell environment variables. So if you depend on (for example) a non system path in your PATH, you'll need to explicitly specify that in env['ENV']['PATH'] for example. For more details take a read through the users guide, manpage and FAQ.
https://scons.org/doc/production/HTML/scons-user.html
https://scons.org/doc/production/HTML/scons-man.html
https://scons.org/faq.html
And you can reach the community directly via our discord server, IRC channel, or users mailing list
Summary
I am trying to understand a complicated chain of Makefiles, in order to get a build to succeed. I narrowed down my problem to this bit in our build script:
INF_RL=`make -f $BUILD_ROOT/Makefile BUILD_ROOT_MAKEFILE= show__BUILD_INF_RL`
$INF_RL/$BUILD_UTILS_RELDIR/BuildAll.sh
$INF_RL is being set to an empty string (or not being set). If I replace the first line with
INF_RL=/foo_rel_linx86/infrastructure_release/v8.0.14
in order to hardcode what I know $INF_RL is supposed to be, then the build goes smoothly. But I want to know how to fix this the proper way.
What I've Tried / Thought
My first thought was that make -f is failing. So I tried it in my shell:
% make -f $BUILD_ROOT/Makefile BUILD_ROOT_MAKEFILE= show__BUILD_INF_RL
% setenv | grep BUILD_ROOT
BUILD_ROOT=/userhome/andrew.cheong/TPS
Indeed, it returned an empty string. But what conclusion could I draw from this? I wasn't sure if the shell was the same thing as the environment / scope in which Make was chaining together its Makefiles. I abandoned this investigation.
Next, I looked into show__BUILD_INF_RL, which seemed to be defined in $BUILD_ROOT/Makefile:
BUILD_ROOT_MAKEFILE = 1
MAKE_DIRS = src
CASE_KITS = tpsIn tpsOut
REQUIRED_VERSIONS = "case.v$(INF_VS)"
all:
## These next 3 rules allows any variable set in this makefile (and therefore
## the included makefile.include to have it's value echoed from the command
## "make show_<variableName>"
## NOTE: the "disp" target is vital as it allows the show_% implicit rule to be
## recognised as such - implicit rules *must* have a target.
show_% := DISPLAY_MACRO = $(#:show_%=%)
show_% : disp
# echo $($(DISPLAY_MACRO))
disp:
include $(BUILD_ROOT)/makefile.include
Here, I faced more questions:
What is BUILD_ROOT_MAKEFILE for? Why is it set to 1, then seemingly something else in the make -f command?
In the make -f command, is BUILD_ROOT_MAKEFILE= its own argument? If so, what kind of target or rule is that? Otherwise, why is it being set to the macro?
In $BUILD_ROOT, there is another file, makefile.LINUX_X86.include:
BUILD_INF_RL = /foo_rel_linx86/infrastructure_release/v$(INF_VS)
$(warning $(BUILD_INF_RL))
BUILD_UTILS = $(BUILD_INF_RL)/build-utils_LINUX_X86
Though a completely ignorant guess, I think BUILD_INF_RL is being set here, and intended to be extracted into the build script's variable INF_RL when the macro show__BUILD_INF_RL is invoked. I added the middle line to see if it was indeed being set, and indeed, I get this output when running the build script:
/userhome/andrew.cheong/TPS/makefile.LINUX_X86.include:3: /foo_rel_linx86/infrastructure_release/v8.0.14
i.e. Looks like what I've hardcoded way above! But why doesn't it make it into INF_RL? There is yet another file, makefile.include, also in $BUILD_ROOT:
#
# INCLUDE THIS FILE AS THE LAST LINE IN THE LOCAL MAKEFILE
#
# makefile.include - use this file to define global build settings
# e.g. infrastructure version and location, or third-party
#
# supported macros in addition to build-utils-makefile.include
#
# BUILD_INF_RL : optional, specification of infrastructure release location
# defaults to vdev_build area
#
include $(BUILD_ROOT)/../../makefile.include.$(BUILD_ARCH).Versions
#include $(BUILD_UTILS)/makefile.archdef.include
include $(BUILD_ROOT)/makefile.$(BUILD_ARCH).include
$(warning $(BUILD_INF_RL))
_BUILD_INF_RL = $(BUILD_INF_RL)
# place the results at the root of the infdemo tree
BUILD_DEST = $(BUILD_ROOT)
INCLUDE_DIRS += $(BUILD_INF_RL)/core/$(BUILD_TARGET)/include
LINK_DIRS += $(BUILD_INF_RL)/core/$(BUILD_TARGET)/lib
# libraries required for a typical fidessa app, including OA and DB access
FIDEVMAPP_LIBS = FidApp FidInf FidCore Fidevm
include $(BUILD_UTILS)/makefile.include
That $(warning ...) is again mine, and when running the build script, I get:
/userhome/andrew.cheong/TPS/makefile.include:18: /foo_rel_linx86/infrastructure_release/v8.0.14
The Question
The fact that both $(warning ...)s show up when I run the build script that's calling the make -f ... show__BUILD_INF_RL, tells me that those Makefiles are being included. Then what is causing the macro to fail and return an empty string instead of the correct INF_RL path?
Historical Notes
These build scripts were written at a time when we were only compiling for Solaris. (The scripts were based on templates written by an infrastructure team that loosely accounted for both Solaris and Linux, but we never ran the Linux branch, as it was unnecessary.) We are now fully migrating to Linux, and hitting this issue. The reason I'm skeptical of it being a Linux versus Solaris issue is that we have at least four other products that use a similar Makefile chain and have been migrated with no issues. Not sure why this one in particular is behaving different.
Your question got very long and complex so I didn't read it all... for SO it's often better if you just ask a specific targeted question that you want to know the answer to, with a simple repro case.
I can't say why different makefiles behave differently, but this line:
show_% := DISPLAY_MACRO = $(#:show_%=%)
seems really wrong to me. This is (a) setting the variable show_%, which don't actually use anywhere, (b) to the simply expanded string DISPLAY_MACRO = because at this point in the makefile the variable $# is not set to any value.
Maybe you wanted this line to be this instead:
show_% : DISPLAY_MACRO = $(#:show_%=%)
(note : not :=) so that it's a pattern-specific variable assignment, not a simple variable assignment?
I use scons (V1.1.0) for a project that contains a build step that involves the flex tool.
The definition for the flex command in the scons default rules is:
env["LEX"] = env.Detect("flex") or "lex"
env["LEXFLAGS"] = SCons.Util.CLVar("")
env["LEXCOM"] = "$LEX $LEXFLAGS -t $SOURCES > $TARGET"
which I don't want to change.
However, since -t causes #line directives to be created in the output file that refer to the file "<stdout>", this confuses the subsequent gcov processing.
As a solution, I found that -o can be used to override the file name flex produces into the #line directives (it still produces its output on stdout due to the -t option which apparently has precedence).
To achieve that, I added this in the project's SConscript file:
env.AppendUnique(LEXFLAGS = ['-o $TARGET','-c'],delete_existing=1)
I added the -c option (which does nothing) only to show the difference between how it is treated compared to -o.
An according debug print in the SConscript file results in the following (as expected):
repr(env["LEXFLAGS"]) = ['-o $TARGET', '-c']
This results in the following command line, according to the scons log:
flex "-o build/myfile.cpp" -c -t src/myfile.ll > build/myfile.cpp
So the -c option gets into the command line as desired, but the -o option and its filename parameter has double quotes around it, that must have been created by scons when expanding the LEXFLAGS variable.
When I use this definition for LEXFLAGS instead:
env.AppendUnique(LEXFLAGS = ['--outfile=$TARGET','-c'],delete_existing=1)
the resulting command line works as desired:
flex --outfile=build/myfile.cpp -c -t src/myfile.ll > build/myfile.cpp
So one could speculate that the blank in the -o case caused the double quotes to be used, maybe in an attempt to bind the content together into one logical parameter for the command.
So while my immediate problem is solved by using --outfile, my question is still is it possible to rid of the double quotes in the -o case?
Thanks,
Andy
SCons 1.1.0 is extremely old at this point. I'd recommend trying 2.3.0. But your analysis is correct; if an option (a single option, that is) has a space in it, SCons will quote it so it stays a single option. But you don't have a single option; you really have two, '-o' and '$TARGET'. Just break it up like that and it'll work.
I'm creating a project and using GNU Autoconf tools to do the configuring and making. I've set up all my library checking and header file checking but can't seem to figure out how to check if an executable exists on the system and fail if it doesn't exist.
I've tried:
AC_CHECK_PROG(TEST,testprogram,testprogram,AC_MSG_ERROR(Cannot find testprogram.))
When I configure it runs and outputs:
Checking for testprogram... find: `testprogram. 15426 5 ': No such file or directory
but does not fail.
I found this to be the shortest approach.
AC_CHECK_PROG(FFMPEG_CHECK,ffmpeg,yes)
AS_IF([test x"$FFMPEG_CHECK" != x"yes"], [AC_MSG_ERROR([Please install ffmpeg before configuring.])])
Try this which is what I just lifted from a project of mine, it looks for something called quantlib-config in the path:
# borrowed from a check for gnome in GNU gretl: def. a check for quantlib-config
AC_DEFUN(AC_PROG_QUANTLIB, [AC_CHECK_PROG(QUANTLIB,quantlib-config,yes)])
AC_PROG_QUANTLIB
if test x"${QUANTLIB}" == x"yes" ; then
# use quantlib-config for QL settings
[.... more stuff omitted here ...]
else
AC_MSG_ERROR([Please install QuantLib before trying to build RQuantLib.])
fi
Similar to the above, but has the advantage of also being able to interact with automake by exporting the condition variable
AC_CHECK_PROG([ffmpeg],[ffmpeg],[yes],[no])
AM_CONDITIONAL([FOUND_FFMPEG], [test "x$ffmpeg" = xyes])
AM_COND_IF([FOUND_FFMPEG],,[AC_MSG_ERROR([required program 'ffmpeg' not found.])])
When using AC_CHECK_PROG, this is the most concise version that I've run across is:
AC_CHECK_PROG(BOGUS,[bogus],[bogus],[no])
test "$BOGUS" == "no" && AC_MSG_ERROR([Required program 'bogus' not found.])
When the program is missing, this output will be generated:
./configure
...cut...
checking for bogus... no
configure: error: Required program 'bogus' not found.
Or when coupled with the built-in autoconf program checks, use this instead:
AC_PROG_YACC
AC_PROG_LEX
test "$YACC" == ":" && AC_MSG_ERROR([Required program 'bison' not found.])
test "$LEX" == ":" && AC_MSG_ERROR([Required program 'flex' not found.])
Stumbled here while looking for this issue, I should note that if you want to have your program just looked in pathm a runtime test is enough:
if ! which programname >/dev/null ; then
AC_MSG_ERROR([Missing programname]
fi
This is not exactly a short approach, it's rather a general purporse approach (although when there are dozens of programs to check it might be also the shortest approach). It's taken from a project of mine (the prefix NA_ stands for “Not Autotools”).
A general purpose macro
dnl ***************************************************************************
dnl NA_REQ_PROGS(prog1, [descr1][, prog2, [descr2][, etc., [...]]])
dnl
dnl Checks whether one or more programs have been provided by the user or can
dnl be retrieved automatically. For each program `progx` an uppercase variable
dnl named `PROGX` containing the path where `progx` is located will be created.
dnl If a program is not reachable and the user has not provided any path for it
dnl an error will be generated. The program names given to this function will
dnl be advertised among the `influential environment variables` visible when
dnl launching `./configure --help`.
dnl ***************************************************************************
AC_DEFUN([NA_REQ_PROGS], [
m4_if([$#], [0], [], [
AC_ARG_VAR(m4_translit([$1], [a-z], [A-Z]), [$2])
AS_IF([test "x#S|#{]m4_translit([$1], [a-z], [A-Z])[}" = x], [
AC_PATH_PROG(m4_translit([$1], [a-z], [A-Z]), [$1])
AS_IF([test "x#S|#{]m4_translit([$1], [a-z], [A-Z])[}" = x], [
AC_MSG_ERROR([$1 utility not found])
])
])
m4_if(m4_eval([$# + 1 >> 1]), [1], [], [NA_REQ_PROGS(m4_shift2($*))])
])
])
Sample usage
NA_REQ_PROGS(
[find], [Unix find utility],
[xargs], [Unix xargs utility],
[customprogram], [Some custom program],
[etcetera], [Et cetera]
)
So that within Makefile.am you can do
$(XARGS)
or
$(CUSTOMPROGRAM)
and so on.
Features
It advertises the programs among the “influential environment variables” visible when the final user launches ./configure --help, so that an alternative path to the program can be provided
A bash variable named with the same name of the program, but upper case, containing the path where the program is located, is created
En error is thrown if any of the programs given have not been found and the user has not provided any alternative path for them
The macro can take infinite (couples of) arguments
When you should use it
When the programs to be tested are vital for compiling your project, so that the user must be able to provide an alternative path for them and an error must be thrown if at least one program is not available at all
When condition #1 applies to more than one single program, in which case there is no need to write a general purpose macro and you should just use your own customized code
I want to inject a "Cleanup" target which depends on a number of other targets finishing before it goes off and gzip's some log files. It's important that I not gzip early as this can cause some of the tools to fail.
How can I inject a cleanup target for Scons to execute?
e.g. I have targets foo and bar. I want to inject a new custom target called 'cleanup' that depends on foo and bar and runs after they're both done, without the user having to specify
% scons foo cleanup
I want them to type:
% scons foo
but have scons execute as though the user had typed
% scons foo cleanup
I've tried creating the cleanup target and appending to sys.argv, but it seems that scons has already processed sys.argv by the time it gets to my code so it doesn't process the 'cleanup' target that I manually append to sys.argv.
you shouldn't use _Add_Targets or undocumented features, you can just add your cleanup target to BUILD_TARGETS:
from SCons.Script import BUILD_TARGETS
BUILD_TARGETS.append('cleanup')
if you use this documented list of targets instead of undocumented functions, scons won't be confused when doing its bookkeeping. This comment block can be found in SCons/Script/__init__.py:
# BUILD_TARGETS can be modified in the SConscript files. If so, we
# want to treat the modified BUILD_TARGETS list as if they specified
# targets on the command line. To do that, though, we need to know if
# BUILD_TARGETS was modified through "official" APIs or by hand. We do
# this by updating two lists in parallel, the documented BUILD_TARGETS
# list, above, and this internal _build_plus_default targets list which
# should only have "official" API changes. Then Script/Main.py can
# compare these two afterwards to figure out if the user added their
# own targets to BUILD_TARGETS.
so I guess it is intended to change BUILD_TARGETS instead of calling internal helper functions
One way is to have the gzip tool depend on the output of the log files. For example, if we have this C file, 'hello.c':
#include <stdio.h>
int main()
{
printf("hello world\n");
return 0;
}
And this SConstruct file:
#!/usr/bin/python
env = Environment()
hello = env.Program('hello', 'hello.c')
env.Default(hello)
env.Append(BUILDERS={'CreateLog':
Builder(action='$SOURCE.abspath > $TARGET', suffix='.log')})
log = env.CreateLog('hello', hello)
zipped_log = env.Zip('logs.zip', log)
env.Alias('cleanup', zipped_log)
Then running "scons cleanup" will run the needed steps in the correct order:
gcc -o hello.o -c hello.c
gcc -o hello hello.o
./hello > hello.log
zip(["logs.zip"], ["hello.log"])
This is not quite what you specified, but the only difference between this example and your requirement is that "cleanup" is the step that actually creates the zip file, so that is the step that you have to run. Its dependencies (running the program that generates the log, creating that program) are automatically calculated. You can now add the alias "foo" as follows to get the desired output:
env.Alias('foo', zipped_log)
In version 1.1.0.d20081104 of SCons, you can use the private internal SCons method:
SCons.Script._Add_Targets( [ 'MY_INJECTED_TARGET' ] )
If the user types:
% scons foo bar
The above code snippet will cause SCons to behave as though the user had typed:
% scons foo bar MY_INJECTED_TARGET