SCons: how to explicitly express the dependencies between envs? - scons

I have a project build by scons.
In the project there are multiple components, including client, shell, engine and etc...
Each component uses different compile options so they are split into different env.
And both shell and engine are going to require client libraries built first.
In the environment settings, both shell and engine has something like "-lclient -L[installpath]/lib", and SConscriptClient is going to build the libclient.a in [installpath]/lib.
So I expect SConscriptClient is run before everything else.
so in the code I have something like:
clientbuild = clientEnv.SConscript ( 'SConscriptClient', variant_dir=clientDir )
if hasShell:
shellbuild = shellEnv.SConscript ( 'SConscriptShell', variant_dir=shellDir )
Depends ( shellbuild, clientbuild )
if hasEngine:
enginebuild = engineEnv.SConscript ( 'SConscriptEngine', variant_dir=engineDir )
Depends ( enginebuild, clientbuild )
However it seems scons is not smart enough to understand the dependencies between client/shell and engine ( that means the Depends call doesn't take effect ). It still try to run SConscriptShell before SConscriptClient
Is there anything i can do to set the dependeices between sconscript?

You shouldn't have to explicitly set these dependencies with Depends(). If the client library is a target built by SCons and both the shell and engine link that library, then SCons should be able to implicitly determine the dependencies and build the client first.
Basically, I see 2 issues here:
Why doesn't SCons implicitly figure out the dependencies?
Why isn't it working as is with the explicit calls to Depends()?
If we figure out number 1, then we wont have to figure out number 2. But just to be complete, I think number 2 isnt working because of what the call to SConscript() is returning. In the subsidiary SConscript scripts (SConscriptClient, SConscriptShell, and SConscriptEngine) are you returning the target? If not, I would imagine the clientbuild variable would be None. To return the target use the Return() SCons function and pass it the return value of the Library() builder.
As for why SCons cant figure out the dependencies implicitly, we would need to see the subsidiary SConscript build scripts. But I can imagine its because you are probably specifying the client library "by hand", so SCons doesnt see the dependency.
The way to build a program with a library so that SCons can see the dependency, you need to use the LIBS construction variable, as follows:
env.Append(LIBS='client')
env.Append(LIBPATH='path/to/client/lib')
env.Program(target='shell', source='shell.cc')
Notice I don't use the -l nor the -L flags above, SCons will add those in a platform independent manner. If you're specifying the library "by hand" by specifying it like this: '-lclient' then SCons wont see the dependency. This is by design, and is more interesting with include paths: if you have a lot of include paths with header files that will almost never change, then (for performance reasons) you don't want SCons to scan them for changes, and thus specify them "by hand".
One additional comment, normally the different environments are passed to the subsidiary SConscript build scripts differently, as follows:
clientEnv = Environment()
# set the clientEnv accordingly
SConscript ('SConscriptClient', variant_dir=clientDir, exports=['clientEnv'] )
SConscriptClient:
Import('clientEnv')
# You may want to clone the clientEnv here, if you want to make
# changes that you don't want seen in the rest of the build

Related

Scons: command-line build variables without restrictions on allowed values

Used Scons version: 3.0.1 (for historic reasons)
I want to add a command-line build variable in my Scons script which is not limited by which values it expects. Both EnumVariable and ListVariable require you to provide a list of allowed values or elements.
I want a variable that (just like a C++ define can be passed to g++ via -Dfoo=bar (or at least -Dfoo_bar) ) can be passed to the Scons script.
Anything like
scons my_define=foo_bar
or
scons my_define=foo=bar,gnarl=argl
or
scons my_define=foo my_define=bar
would work.
I did not find anything like that in the user guide or searching via Google.
Is this possible? If yes, how?
Yes. It's in the users guide at
https://scons.org/doc/production/HTML/scons-user.html#idp140637539360912
While there are validated variable types provided by the various vars.Add*() methods. You can have a non-validated variable by just using plain vars.Add()
Here's an example from the manpage.
vars.Add('CC', help='The C compiler')

Build multiple targets with SCONS

I'm building a modular project with SCONS.
I'd like to have multiple SW products to be compiled as a colleciton of multiple SW subcomponents:
Eg:
*Sconstruct:
Sconscript("SW_PRODUCT_1.Sconscript")
Sconscript("SW_PRODUCT_2.Sconscript")
*With SW_PRODUCT_1.Sconscript:
Sconscript("COMPONENT_A.Sconscript")
Sconscript("COMPONENT_B.Sconscript")
Sconscript("COMPONENT_C.Sconscript")
*And with SW_PRODUCT_2.Sconscript:
Sconscript("COMPONENT_A.Sconscript")
Sconscript("COMPONENT_B.Sconscript")
ie I'd like the Builder compile once each component as a Library, and then each SW_PRODUCT take the list of components it wants to build an executable.
But I have the next problem , the SCONS tool returns the next error:
"scons: *** Two environments with different actions were specified for the same target"
This is truth since according to the script the components A and B are invoked twice with exactly the same target names but I hope the SCONS would handle the fact that multiple Targets can share libraries. I expected the SCONS to realize in the second call to COMPONENTS_A and B that the target "up to date" but it does not work that way.
I've though about a workaround compiling two diferent Libraries (different name) for each PRODUCT but that is not what I want since the Libraries are exactly the same and I don't want to waste compilation time to build exactly the same N times. I would like just to build the same pool of libraries and then pick such libraries on demand with each product.
Is there any easy way to handle this architecture with Scons?
Simply put all (!) required libraries (=components) and products into the same top-level SConstruct file. You can use SConscript calls to include the build definitions from your different subfolders, but don't leave any of them out.
Try to imagine that you want to build everything at once. Don't try to setup build dependencies (or a build order) by how you include SConscripts for "components" in your build files, or not.
This is important, because it ensures that SCons can see the full source tree with all sources, libraries and programs that you might ever need. Then it is able to figure out the build dependencies all by itself.
In the next step, simply define combined targets for your products via the Alias command, i.e.:
env = Environment()
program_a = env.Program('maina', Glob('*.cpp'), LIBS=['compx','compy'])
env.Alias('producta', program_a)
(see also chap. 25 "Alias Targets" in the User Guide). Now you are able to call
scons producta
and SCons will figure out for you, which components are up-to-date or need a rebuild. If you then specify multiple targets
scons producta productb productk
the required libs will only be built once and are shared among the products automatically.
In our SCons Wiki:Recipes you can also find some more examples and setups to give you an impression about how other people setup their projects.

Scons command/explicit dependency

I have a code snippet similar to this:
# Compile protobuf headers
env.Protoc(...)
# Move headers to 'include' (compiled via protobuf)
env.Command([include headers...], [headers...], move_func)
# Compile program (depends on 'include' files)
out2 = SConscript('src/SConscript')
Depends(out2, [include headers...])
Basically, I have Protoc() compiling protobuf files, then the headers are moved to the 'include' directory by env.Command() and finally the program is compiled through a SConscript file in the 'src'.
Since these are header files that are being moved (that the src compilation depends on), they are not explicitly defined as a dependency by scons (as far as I understand). Thus, the compilation runs, but the header files haven't been moved so it fails. I have tried exposing the dependency via Depends() and Requires() without success.
I understand that in the usual case, scons should "figure-out" dependencies, but I don't know how it could do that here.
Thanks!
You seem to be thinking in "make" ways about your build process, which is the wrong approach when using SCons. You can't order single build steps by putting them in different SConscripts, and then including those in a special order. You have to define proper dependencies between your actual sources (C/CPP files for example) and a target like a program or PDF file. Then SCons is able to figure out the correct build order, and will traverse through the folder structure of your project automatically. If required, it will enter subfolders more than once when the dependency graph (DAG) dictates this. Defining this kind of dependencies between inputs and outputs is usually done, using a Builder...and in your case the Install() builder would be a good fit. Please also regard the hints for #2 in the list of "most frequently-asked FAQs" ( https://bitbucket.org/scons/scons/wiki/FrequentlyAskedQuestions).
Further, I can only recommend to read a little more in the UserGuide ( http://www.scons.org/doc/production/HTML/scons-user.html ) to get a better feeling for how to do things in a more "SConsy" way. If you get stuck, feel free to ask further questions on our mailing list at scons-users#scons.org (see http://www.scons.org/lists.php ).
Finally, if you have a lot of steps that you want to execute in serial, and that don't require any special input/output files, SCons is probably not the right tool for your current task. It's designed as a file-oriented build system with automatic parallelization in mind, a simple (Python?) script might be better at the mere serial stuff...

How to prevent scons from cleaning parent and sibling directories?

I'm working on implementing a build system using scons for a somewhat large software project. There is a directory structure which separates the code for individual libraries and programs into their own directories. With our existing make system, I can do a "make clean" in a single program directory and it will only clean the files associated with the source in that directory. If I do an "scons -c" though, it recognizes that the program depends on a slew of libraries that are in sibling (or cousin) directories and cleans all of the files for those as well. This is not what I want since I then have to rebuild all of these libraries which can take several minutes.
I have tried playing with the "NoClean()" command, but have not gotten it to work in the way I need. Given the size of the code base and complexity of the directory structure, I can't realistically have a NoClean() line for every file in every library.
Is there any way to tell scons to ignore any dependencies above the current directory when doing a clean (i.e. scons -c) ?
I'd love to have a good answer to this myself.
The only solution that I can offer for now is that you get Noclean working.
So in your library, you should have something like this
lib_objs = SharedObject(source_list)
mylib = SharedLibrary('libname', lib_objs)
So for this we want to protect the library and the sources from being cleaned.
NoClean([mylib, lib_objs])
Notice that I had to split the building of the object files from the library because I want to be able to pass them to NoClean as well.
Try using the target name when cleaning.
scons -c aTargetName
You can use the SCons Alias() function to simplify the target name and to also group several target names into one alias.
With this approach you'll have to add an alias in each appropriate subdir, which isn't necessarily a bad thing :)

How to test SCons modifications

My team is creating a build system based on SCons. We have created a bunch of helper classes in our own site_scons/site_tools folder.
My task is to create and run tests on our code, using pyunit. The test code would probably live in a subfolder, with the directory layout looking something like:
SConstruct
our_source_code/
Sconscript
site_scons/
site_tools/
a.py
b.py
c.py
tests/
test_a.py
test_b.py
test_c.py
Now my question is: What is the best way to invoke our tests, given they will probably require the correct SCons environment set up? (that is a.py uses SCons.Environment)
Do I add a Builder or a Command? Or something else?
I think the best approach would be to use the test setup code from SCons itself. This requires a SVN checkout of SCons, as the test files are not shipped with the regular SCons tarballs. This is probably workable, as not everyone in your team would be writing tools and running tests on them.
For example, this is the test for javac. Basically you write out files that you want, run a SConstruct, then check the results are what you expected. You can mock tools with Python scripts, to ensure they are really called with the flags and files that you expect. For example:
import TestSCons
test = TestSCons.TestSCons()
test.write('SConstruct', '''env = Environment(tools = ["yourtool"])
env.RunYourTool()''')
test.write('sourcefile.x', 'Content goes here')
test.run(arguments = '.', stderr = None)
test.must_match('outputfile', 'desired contents')
test.pass_test()
There are also more instructions on writing SCons tools tests on the swtoolkit wiki, which is a seemingly-defunct SCons extension from Google. The info on the wiki is still useful, and there are some good examples on how to write tests for custom SCons tools.

Resources