I'm building a modular project with SCONS.
I'd like to have multiple SW products to be compiled as a colleciton of multiple SW subcomponents:
Eg:
*Sconstruct:
Sconscript("SW_PRODUCT_1.Sconscript")
Sconscript("SW_PRODUCT_2.Sconscript")
*With SW_PRODUCT_1.Sconscript:
Sconscript("COMPONENT_A.Sconscript")
Sconscript("COMPONENT_B.Sconscript")
Sconscript("COMPONENT_C.Sconscript")
*And with SW_PRODUCT_2.Sconscript:
Sconscript("COMPONENT_A.Sconscript")
Sconscript("COMPONENT_B.Sconscript")
ie I'd like the Builder compile once each component as a Library, and then each SW_PRODUCT take the list of components it wants to build an executable.
But I have the next problem , the SCONS tool returns the next error:
"scons: *** Two environments with different actions were specified for the same target"
This is truth since according to the script the components A and B are invoked twice with exactly the same target names but I hope the SCONS would handle the fact that multiple Targets can share libraries. I expected the SCONS to realize in the second call to COMPONENTS_A and B that the target "up to date" but it does not work that way.
I've though about a workaround compiling two diferent Libraries (different name) for each PRODUCT but that is not what I want since the Libraries are exactly the same and I don't want to waste compilation time to build exactly the same N times. I would like just to build the same pool of libraries and then pick such libraries on demand with each product.
Is there any easy way to handle this architecture with Scons?
Simply put all (!) required libraries (=components) and products into the same top-level SConstruct file. You can use SConscript calls to include the build definitions from your different subfolders, but don't leave any of them out.
Try to imagine that you want to build everything at once. Don't try to setup build dependencies (or a build order) by how you include SConscripts for "components" in your build files, or not.
This is important, because it ensures that SCons can see the full source tree with all sources, libraries and programs that you might ever need. Then it is able to figure out the build dependencies all by itself.
In the next step, simply define combined targets for your products via the Alias command, i.e.:
env = Environment()
program_a = env.Program('maina', Glob('*.cpp'), LIBS=['compx','compy'])
env.Alias('producta', program_a)
(see also chap. 25 "Alias Targets" in the User Guide). Now you are able to call
scons producta
and SCons will figure out for you, which components are up-to-date or need a rebuild. If you then specify multiple targets
scons producta productb productk
the required libs will only be built once and are shared among the products automatically.
In our SCons Wiki:Recipes you can also find some more examples and setups to give you an impression about how other people setup their projects.
Related
I work with the Yocto Project quite a bit and a common challenge is determining why (or from what recipe) a file has been included on the rootfs. This is something that can hopefully be derived from the build system's environment, log & meta data. Ideally, a set of commands would allow linking a file back to a source (ie. recipe).
My usual strategy is to perform searches on the meta data (e.g. grep -R filename ../layers/*) and searches on the internet of said filenames to find clues of possible responsible recipes. However, this is not always very effective. In many cases, filenames are not explicitly stated within a recipe. Additionally, there are many cases where a filename is provided by multiple recipes which leads to additional work to find which recipe ultimately supplied it. There are of course many other clues available to find the answer. Regardless, this investigation is often quite laborious when it seems the build system should have enough information to make resolving the answer simple.
This is exact use case for oe-pkgdata-util script and its subcommand find-path. That script is part of openembedded-core.
See this example (executed in OE build environment, i.e. bitbake works):
tom#pc:~/oe/build> oe-pkgdata-util find-path /lib/ld-2.24.so
glibc: /lib/ld-2.24.so
You can clearly see that this library belongs to glibc recipe.
oe-pkgdata-util has more useful subcommands to see information about packages and recipes, it worth to check the --help.
If you prefer a graphical presentation, the Toaster web UI will also show you this, plus dependency information.
The candidate files deployed for each recipe are placed in each $WORKDIR/image
So you can cd to
$ cd ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}
and perform a
$ find . -path '*/image/*/fileYouAreLookingFor'
from the result you should be able to infer the ${PN} of the recipe which deploys such file.
For example:
$ find . -path '*/image/*/mc'
./bash-completion/2.4-r0/image/usr/share/bash-completion/completions/mc
./mc/4.8.18-r0/image/usr/share/mc
./mc/4.8.18-r0/image/usr/bin/mc
./mc/4.8.18-r0/image/usr/libexec/mc
./mc/4.8.18-r0/image/etc/mc
I have a code snippet similar to this:
# Compile protobuf headers
env.Protoc(...)
# Move headers to 'include' (compiled via protobuf)
env.Command([include headers...], [headers...], move_func)
# Compile program (depends on 'include' files)
out2 = SConscript('src/SConscript')
Depends(out2, [include headers...])
Basically, I have Protoc() compiling protobuf files, then the headers are moved to the 'include' directory by env.Command() and finally the program is compiled through a SConscript file in the 'src'.
Since these are header files that are being moved (that the src compilation depends on), they are not explicitly defined as a dependency by scons (as far as I understand). Thus, the compilation runs, but the header files haven't been moved so it fails. I have tried exposing the dependency via Depends() and Requires() without success.
I understand that in the usual case, scons should "figure-out" dependencies, but I don't know how it could do that here.
Thanks!
You seem to be thinking in "make" ways about your build process, which is the wrong approach when using SCons. You can't order single build steps by putting them in different SConscripts, and then including those in a special order. You have to define proper dependencies between your actual sources (C/CPP files for example) and a target like a program or PDF file. Then SCons is able to figure out the correct build order, and will traverse through the folder structure of your project automatically. If required, it will enter subfolders more than once when the dependency graph (DAG) dictates this. Defining this kind of dependencies between inputs and outputs is usually done, using a Builder...and in your case the Install() builder would be a good fit. Please also regard the hints for #2 in the list of "most frequently-asked FAQs" ( https://bitbucket.org/scons/scons/wiki/FrequentlyAskedQuestions).
Further, I can only recommend to read a little more in the UserGuide ( http://www.scons.org/doc/production/HTML/scons-user.html ) to get a better feeling for how to do things in a more "SConsy" way. If you get stuck, feel free to ask further questions on our mailing list at scons-users#scons.org (see http://www.scons.org/lists.php ).
Finally, if you have a lot of steps that you want to execute in serial, and that don't require any special input/output files, SCons is probably not the right tool for your current task. It's designed as a file-oriented build system with automatic parallelization in mind, a simple (Python?) script might be better at the mere serial stuff...
I have a project build by scons.
In the project there are multiple components, including client, shell, engine and etc...
Each component uses different compile options so they are split into different env.
And both shell and engine are going to require client libraries built first.
In the environment settings, both shell and engine has something like "-lclient -L[installpath]/lib", and SConscriptClient is going to build the libclient.a in [installpath]/lib.
So I expect SConscriptClient is run before everything else.
so in the code I have something like:
clientbuild = clientEnv.SConscript ( 'SConscriptClient', variant_dir=clientDir )
if hasShell:
shellbuild = shellEnv.SConscript ( 'SConscriptShell', variant_dir=shellDir )
Depends ( shellbuild, clientbuild )
if hasEngine:
enginebuild = engineEnv.SConscript ( 'SConscriptEngine', variant_dir=engineDir )
Depends ( enginebuild, clientbuild )
However it seems scons is not smart enough to understand the dependencies between client/shell and engine ( that means the Depends call doesn't take effect ). It still try to run SConscriptShell before SConscriptClient
Is there anything i can do to set the dependeices between sconscript?
You shouldn't have to explicitly set these dependencies with Depends(). If the client library is a target built by SCons and both the shell and engine link that library, then SCons should be able to implicitly determine the dependencies and build the client first.
Basically, I see 2 issues here:
Why doesn't SCons implicitly figure out the dependencies?
Why isn't it working as is with the explicit calls to Depends()?
If we figure out number 1, then we wont have to figure out number 2. But just to be complete, I think number 2 isnt working because of what the call to SConscript() is returning. In the subsidiary SConscript scripts (SConscriptClient, SConscriptShell, and SConscriptEngine) are you returning the target? If not, I would imagine the clientbuild variable would be None. To return the target use the Return() SCons function and pass it the return value of the Library() builder.
As for why SCons cant figure out the dependencies implicitly, we would need to see the subsidiary SConscript build scripts. But I can imagine its because you are probably specifying the client library "by hand", so SCons doesnt see the dependency.
The way to build a program with a library so that SCons can see the dependency, you need to use the LIBS construction variable, as follows:
env.Append(LIBS='client')
env.Append(LIBPATH='path/to/client/lib')
env.Program(target='shell', source='shell.cc')
Notice I don't use the -l nor the -L flags above, SCons will add those in a platform independent manner. If you're specifying the library "by hand" by specifying it like this: '-lclient' then SCons wont see the dependency. This is by design, and is more interesting with include paths: if you have a lot of include paths with header files that will almost never change, then (for performance reasons) you don't want SCons to scan them for changes, and thus specify them "by hand".
One additional comment, normally the different environments are passed to the subsidiary SConscript build scripts differently, as follows:
clientEnv = Environment()
# set the clientEnv accordingly
SConscript ('SConscriptClient', variant_dir=clientDir, exports=['clientEnv'] )
SConscriptClient:
Import('clientEnv')
# You may want to clone the clientEnv here, if you want to make
# changes that you don't want seen in the rest of the build
I'm working on implementing a build system using scons for a somewhat large software project. There is a directory structure which separates the code for individual libraries and programs into their own directories. With our existing make system, I can do a "make clean" in a single program directory and it will only clean the files associated with the source in that directory. If I do an "scons -c" though, it recognizes that the program depends on a slew of libraries that are in sibling (or cousin) directories and cleans all of the files for those as well. This is not what I want since I then have to rebuild all of these libraries which can take several minutes.
I have tried playing with the "NoClean()" command, but have not gotten it to work in the way I need. Given the size of the code base and complexity of the directory structure, I can't realistically have a NoClean() line for every file in every library.
Is there any way to tell scons to ignore any dependencies above the current directory when doing a clean (i.e. scons -c) ?
I'd love to have a good answer to this myself.
The only solution that I can offer for now is that you get Noclean working.
So in your library, you should have something like this
lib_objs = SharedObject(source_list)
mylib = SharedLibrary('libname', lib_objs)
So for this we want to protect the library and the sources from being cleaned.
NoClean([mylib, lib_objs])
Notice that I had to split the building of the object files from the library because I want to be able to pass them to NoClean as well.
Try using the target name when cleaning.
scons -c aTargetName
You can use the SCons Alias() function to simplify the target name and to also group several target names into one alias.
With this approach you'll have to add an alias in each appropriate subdir, which isn't necessarily a bad thing :)
How do I link ".a" fat-static-libs with Apportable?
In the beginning, I did get a warning,
Warning: Library not found for lib-name. Try adding this to the
'deps' array in the 'add_params' section of your configuration.json
file. Check ~/.apportable/SDK/System for the directory names that
correspond to 'deps' entries.
I did add it to "deps" in the JSON right where it says this:
// A list of dependencies. Typically these correspond to frameworks in the xcode project.
My library isn't a framework, though. It's just a .a static library that has armv7, armv7s, and i386 parts which I assembled using lipo from two libraries (an armv7 and armv7s .a and a i386 .a) built with Xcode. They both use a single framework which is this Boost framework.
Adding it under deps squelched that warning message, but the apportable linker is still giving undefined references, so it is still not properly linking this file.
Now I know that Apportable has to re-jitter all this stuff, since Android won't know what to do with a Mac-executable format, so it's probably got to go pick apart my library and possibly turn it into an ELF-library before final linking. I'm not sure how to go about debugging this at this point, but is this supported at all?
Since you get "library not found" that means Apportable simply can't find the file. Hence the problem is merely with the file's location (or existence) and not what's in the library or how it is built.
The most common issue with dependent targets, especially Xcode projects dropped into other Xcode projects respectively workspaces with multiple projects, is that the resulting output of each target/project goes to different folders. Then Apportable (as well as xcodebuild under some circumstances) can't find the resulting libraries.
First step you should try is to make sure that the target dependencies are set. Select the app target, go to the Build Phases pane and under Target Dependencies add all frameworks and libraries that appear in the list and that your project depends on. This should ensure that the dependent frameworks/libraries do get built - because Xcode's built-in automatic dependency resolution isn't available to command line tools from what I understand. So you need to explicitly specify the dependent projects respectively their output.
If that doesn't help, you can force all targets to write their output to the same folder. Under Build Settings for every target change the Build Products Path (symbolic name: SYMROOT) to the same folder, for example ~/myprojectsbuildoutput
That way even dependent other projects will place their output in the same folder and xcodebuild as well as Apportable (it depends on xcodebuild) will be able to find the library files.
Tip: make sure your project builds successfully on the command line. Open Terminal, cd into the folder where the .xcodeproj bundle is and enter xcodebuild. If xcodebuild fails merely due to validation, disable Validate Built Product under Build Settings of the app target. If xcodebuild fails, Apportable likely isn't going to work either because it depends on xcodebuild. So as a prerequisite make sure that xcodebuild works on your project.
If xcodebuild also gives you "library not found" try calling it with a specific SYMROOT:
xcodebuild SYMROOT=~/myprojectsbuildoutput
If that then works you know you have to update each target's Build Projects Path. From what I know it's not currently possible (or not documented) to pass custom xcodebuild parameters via Apportable, so it needs to be set up in the .xcodeproj itself.