Call function inside SCons Builder's action - scons

I'm trying to write a Builder in SCons to call a command line executable with some arguments that are generated by a (series of) python functions.
cmdVars = Variables(None, ARGUMENTS)
cmdVars.AddVariables(
EnumVariable('DEBUG', 'help for debug', 'a', allowed_values=('a','b','c')),
PathVariable('CLI', 'path to cli exe', 'C:\...\blah.exe', PathVariable.PathIsFile)
)
env = Environment(variables = cmdVars)
def generateSomeExtraBitsDependingOnFlag(debug):
if (debug == 'a'):
return "-DDEBUG -DBlah myTextHere"
return ''
myBuilder = Builder(
action = '"$CLI" generateSomeExtraBitsDependingOnFlag("$DEBUG")'
)
<extra stuff to add myBuilder to env and call env.myBuilder>
The "$CLI" input is correctly substituted to output (when calling scons) something like C:\Program Files\...\blah.exe but the output from the function never appears, regardless of the DEBUG setting.
Prepending print(debug) inside the function prints $DEBUG whilst parsing the SConstruct file (so it's not surprising it doesn't match the if condition).
Do I just need to use a Generator or follow the instructions in chapter 18.4 (Builders That Execute Python Functions) to make this work? Is the section Writing Builders That Execute External Commands not what I want here?
I suspect that given understanding, the user guide is clear, but without already knowing the answer, the guide is a little opaque to me.

Try this:
cmdVars = Variables(None, ARGUMENTS)
cmdVars.AddVariables(
EnumVariable('DEBUG', 'help for debug', 'a', allowed_values=('a','b','c')),
PathVariable('CLI', 'path to cli exe', 'C:\...\blah.exe', PathVariable.PathIsFile)
)
env = Environment(variables = cmdVars)
def generateSomeExtraBitsDependingOnFlag(source, target, env, for_signature):
if (env['DEBUG'] == 'a'):
return "-DDEBUG -DBlah myTextHere"
return ''
env['generateSomeExtraBitsDependingOnFlag'] =generateSomeExtraBitsDependingOnFlag
myBuilder = env.Builder(
action = '"$CLI" ${generateSomeExtraBitsDependingOnFlag}'
)
env.Append(BUILDERS = {'myBuilder' : myBuilder})
env.myBuilder('dummy','input')

Related

Using Groovy, how can we call a python script, in windows, with one argument

I need to run a python script in windows system using groovy script.
Example:
python.exe c:/main.py argument1
I am new to groovy and I don't know, how to do it.
Please share me groovy syntax to run python as mentioned in the above example
I am preparing this script for jenkins.
so, "command".execute() is the right start.
But this command only starts a thread and you don't wait for the result.
try this code:
def task = "python main.py".execute()
task.waitFor()
println task.text
These lines start the execution, wait for it to finish and print the result.
To output already during execution for longer running tasks, I've written myself a small helper:
String.metaClass.executeCmd = { silent ->
//make sure that all paramters are interpreted through the cmd-shell
//TODO: make this also work with *nix
def p = "cmd /c ${delegate.value}".execute()
def result = [std: '', err: '']
def ready = false
Thread.start {
def reader = new BufferedReader(new InputStreamReader(p.in))
def line = ""
while ((line = reader.readLine()) != null) {
if (silent != false) {
println "" + line
}
result.std += line + "\n"
}
ready = true
reader.close()
}
p.waitForOrKill(30000)
def error = p.err.text
if (error.isEmpty()) {
return result
} else {
throw new RuntimeException("\n" + error)
}
}
This defines through meta programming a new method on String called executeCmd.
Put this on top of your file and then your line
"python c:/main.py".executeCmd()
This should show you all output during execution and it will help you to handle the paramaters the correcct way through the "cmd /c"-prefix. (If you just call execute on a string, you often run into problems with spaces and other characters in your command.
If you already have the parameters as a list and need some code which also runs on a *nix machine, try to call execute() on a list:
["python", "c:/main.py"].execute()
hope this helps
ps: http://mrhaki.blogspot.com/2009/10/groovy-goodness-executing-string-or.html

How to embed file into template using Scons and Substfile?

I'm trying to create a Substfile rule that will expand a key to the transformed contents of another file. I'm not clear on the setup here to ensure the source file is registered as a dependency.
Logically I want something like:
out = env.Substfile( 'file.in', SUBST_DICT = {
'%SOME_CONTENT%': transform( readfile('depends.txt') ),
}
I'm using a combination of Action and Command to do what I want. I ended up not using Substfile, though it could probably be chained to the command.
This RawStringIt action loads a text file and emits a C++ encoded raw string for the content.
def RawStringIt(varName):
def Impl(target, source, env):
content = source[0].get_text_contents()
with open(target[0].get_path(), 'w') as target_file:
target_file.write( "std::string {} = R\"~~~~({})~~~~\";".format(varName,content))
return 0
return Action(Impl, "creating C++ Raw String $TARGET from $SOURCE" )
base_leaf = env.Command( 'include/runner/base.leaf.hpp', '../share/base.leaf', RawStringIt("dataBaseLeaf") )

Read scons build variables from a external.py file

I want to define the scons build variables in external.py file like
external.py
mode=debug
toolchain=x86
This I want to read back these variables in the SConstruct file which is there in the same directory. Depending on the variable values I want to do some operations!
vars = Variables('external.py')
vars.Add('mode', 'Set the mode for debug or release', 'debug')
if ${RELEASE}=="debug"
#Do these!
elif ${RELEASE}=="release"
#Do that!
If external.py contains valid Python code then you can simply import it using the import keyword. You can then use the dir function to iterate over the names defined in the external module and add them to the SCons variables. You might also want to take a look at the getattr function.
The thing you are missing is Scons Environment with your variables.
vars = Variables('external.py')
vars.Add('mode', 'Set the mode for debug or release', 'debug')
env = Environment(variables = vars)
if env['mode'] == 'debug':
# do action1
elif env['mode'] == 'release':
# do action2
else:
# do action3
You can read more about using Scons here, and about your question here
Soumyajit answer is great but I would add that if you want to be able to override values from your file with the command line and restrain the allowed values for your variables you can do as follow:
# Build variables are loaded in this order:
# Command Line (ARGUMENTS) >> Config File (external.py) >> Default Value
vars = Variables(files='external.py', args=ARGUMENTS)
vars.Add(EnumVariable('mode', 'Build mode.', 'debug', allowed_values=('debug', 'release')))
env = Environment(variables = vars)
if env['mode'] == 'debug':
env.Append(CCFLAGS = [ '-g' ])
# whatever...
else:
env.Append(CCFLAGS = '-O2')
# whatever...
You can invoke you build script like this scons but also override specific variables without editing your config file by doing scons mode=release
If you specify a bad value for your variable you will get an error from Scons like:
$> scons mode=foo
scons: Reading SConscript files ...
scons: *** Invalid value for option mode: foo. Valid values are: ('debug', 'release')

How can I add the build version to a scons build

At the moment I'm using some magic to get the current git revision into my scons builds.. I just grab the version a stick it into CPPDEFINES.
It works quite nicely ... until the version changes and scons wants to rebuild everything, rather than just the files that have changed - becasue the define that all files use has changed.
Ideally I'd generate a file using a custom builder called git_version.cpp and
just have a function in there that returns the right tag. That way only that one file would be rebuilt.
Now I'm sure I've seen a tutorial showing exactly how to do this .. but I can't seem to track it down. And I find the custom builder stuff a little odd in scons...
So any pointers would be appreciated...
Anyway just for reference this is what I'm currently doing:
# Lets get the version from git
# first get the base version
git_sha = subprocess.Popen(["git","rev-parse","--short=10","HEAD"], stdout=subprocess.PIPE ).communicate()[0].strip()
p1 = subprocess.Popen(["git", "status"], stdout=subprocess.PIPE )
p2 = subprocess.Popen(["grep", "Changed but not updated\\|Changes to be committed"], stdin=p1.stdout,stdout=subprocess.PIPE)
result = p2.communicate()[0].strip()
if result!="":
git_sha += "[MOD]"
print "Building version %s"%git_sha
env = Environment()
env.Append( CPPDEFINES={'GITSHAMOD':'"\\"%s\\""'%git_sha} )
You don't need a custom Builder since this is just one file. You can use a function (attached to the target version file as an Action) to generate your version file. In the example code below, I've already computed the version and put it into an environment variable. You could do the same, or you could put your code that makes git calls in the version_action function.
version_build_template="""/*
* This file is automatically generated by the build process
* DO NOT EDIT!
*/
const char VERSION_STRING[] = "%s";
const char* getVersionString() { return VERSION_STRING; }
"""
def version_action(target, source, env):
"""
Generate the version file with the current version in it
"""
contents = version_build_template % (env['VERSION'].toString())
fd = open(target[0].path, 'w')
fd.write(contents)
fd.close()
return 0
build_version = env.Command('version.build.cpp', [], Action(version_action))
env.AlwaysBuild(build_version)

How to create a symbolic link with SCons?

I'm using SCons for building a project and need to add a symbolic link to a file it is installing via env.Install. What command(s) will make a link that's the equivalent of running ln -s on the command line?
SCons doesn't have a dedicated symbolic link command, but you can use os.symlink(src, dst) from Python's os module:
import os
env = Environment()
def SymLink(target, source, env):
os.symlink(os.path.abspath(str(source[0])), os.path.abspath(str(target[0])))
env.Command("file.out", "file.in", SymLink)
This may not work correctly on Windows, I've only tried it on Linux.
There seems to be little advancement in the SCons core code for symbolic link support and I wasn't satisfied any one solution I found on the web. Here is a potential builder which incorporates aspects of both Nick's and richq's answers. Additionally, it will catch name changes (due to the emitter method) and is as platform-agnostic as I could get it.
I prefer this builder because it will make links relative to the directory in which they are installed. One could add an option to force the link to be absolute I suppose, but I have not needed or wanted that yet.
Currently, if the OS doesn't support symlinks, I just pass and do nothing, but one could use os.copytree() for example however the dependency becomes messy if the source is a directory so the emitter would need to do something fancy. I'm up for any suggestions here.
One can put the following code into the file site_scons/site_tools/symlink.py (with blank _init_.py files in the appropriate places). Then do this in the SConstruct file:
SConstruct:
env = Environment()
env.Tool('symlink')
env.SymLink('link_name.txt', 'real_file.txt')
symlink.py:
import os
from os import path
from SCons.Node import FS
from SCons.Script import Action, Builder
def generate(env):
'''
SymLink(link_name,source)
env.SymLink(link_name,source)
Makes a symbolic link named "link_name" that points to the
real file or directory "source". The link produced is always
relative.
'''
bldr = Builder(action = Action(symlink_builder,symlink_print),
target_factory = FS.File,
source_factory = FS.Entry,
single_target = True,
single_source = True,
emitter = symlink_emitter)
env.Append(BUILDERS = {'SymLink' : bldr})
def exists(env):
'''
we could test if the OS supports symlinks here, or we could
use copytree as an alternative in the builder.
'''
return True
def symlink_print(target, source, env):
lnk = path.basename(target[0].abspath)
src = path.basename(source[0].abspath)
return 'Link: '+lnk+' points to '+src
def symlink_emitter(target, source, env):
'''
This emitter removes the link if the source file name has changed
since scons does not seem to catch this case.
'''
lnk = target[0].abspath
src = source[0].abspath
lnkdir,lnkname = path.split(lnk)
srcrel = path.relpath(src,lnkdir)
if int(env.get('verbose',0)) > 3:
ldir = path.relpath(lnkdir,env.Dir('#').abspath)
if rellnkdir[:2] == '..':
ldir = path.abspath(ldir)
print ' symbolic link in directory: %s' % ldir
print ' %s -> %s' % (lnkname,srcrel)
try:
if path.exists(lnk):
if os.readlink(lnk) != srcrel:
os.remove(lnk)
except AttributeError:
# no symlink available, so we remove the whole tree? (or pass)
#os.rmtree(lnk)
print 'no os.symlink capability on this system?'
return (target, source)
def symlink_builder(target, source, env):
lnk = target[0].abspath
src = source[0].abspath
lnkdir,lnkname = path.split(lnk)
srcrel = path.relpath(src,lnkdir)
if int(env.get('verbose',0)) > 4:
print 'target:', target
print 'source:', source
print 'lnk:', lnk
print 'src:', src
print 'lnkdir,lnkname:', lnkdir, lnkname
print 'srcrel:', srcrel
if int(env.get('verbose',0)) > 4:
print 'in directory: %s' % path.relpath(lnkdir,env.Dir('#').abspath)
print ' symlink: %s -> %s' % (lnkname,srcrel)
try:
os.symlink(srcrel,lnk)
except AttributeError:
# no symlink available, so we make a (deep) copy? (or pass)
#os.copytree(srcrel,lnk)
print 'no os.symlink capability on this system?'
return None
This creates a builder to perform the job:
mylib = env.SharedLibrary("foobar", SRCS)
builder = Builder(action = "ln -s ${SOURCE.file} ${TARGET.file}", chdir = True)
env.Append(BUILDERS = {"Symlink" : builder})
mylib_link = env.Symlink("_foobar.so", mylib)
env.Default(mylib)
env.Default(mylib_link)
Again, this solution is for Linux.
If you wanted to issue the command directly to the shell and know the OS, subprocess can be used as well.
E.g.: subprocess.call(['ln', '-s', '</src/path>', '</dest/path>'])
In addition to Nicks solution, you can add a directory symlink by using a file as a directory name carrier. It's not the cleanest solution and debugging path names is a pain, but this works well:
def symlink_last(target_source_env):
src = os.path.basename(os.path.dirname(str(source[0])))
link = "deliverables/last"
print "Symlinking "+ src + "as" + link
os.symlink(src, link)
BUILD_TARGETS.append('link')
install_dir = "deliverables/subdir"
carrier_file = "filename"
builder = Builder(action = symlink_last, chdir=False)
env.Append(BUILDERS={ "Symlink" : builder })
env.Alias(target="link", source=env.Symlink(dir="deliverables", source = install_dir + carrier_file)
This will make a link to deliverables/subdir named deliverables/last, provided that a file deliverables/subdir/filename exists.

Resources