I have a scons project with many SConscript files in the subdirectories. In one of those SConscript files I want to check if particular library is installed on the build host. Here is code snippet:
Import("env")
conf = Configure(env)
if conf.CheckLibWithHeader(...):
doSomething()
env.Library(...)
When I execute build scons fails with strnge error without error message like this:
scons: ***
File "/path/to/SConscript", line 3, in <module>
Line 3 is where I call Configure(). I guess I am doing something not allowed with Configure() function but I cannot find any explanation in scons documentation.
Please help me to debug this.
So the problem was in one of the sibling SConscript files. There was a call to Configure(env) but configuration object was not disposed with conf.Finish() call.
This was an issue because SCons does not allow to create more than one configuration object at any moment. I found this rule by analysing SCons sources - I don't know if this is documented.
Thus when I made my call to Configure(env) SCons thrown an exception because it seen there is another active configuration object orphaned by sibling SConscript.
Related
I'm working on adding googletest unit testing to a large project.
We have a top level SConstruct, which calls SConscripts for each of the git submodules, which may further call other SConstructs contained in each component.
One of these SConscripts will build the googletest framework, which will create the 'gtest.h' file among others. I need to include 'gtest.h' in my source file (test1.cpp) which is in a different repo that has it's own SConscript.
How can I require the gtest.h in test SConscript file before test1.cpp consumes it?
Imagine something like:
Top level SConstruct:
env.SConscript('{path to Gtest}/SConscript')
env.SConscript('{path to my unit test}/SConscript')
SConscript for building Gtest:
env2 = env.Clone()
def buildGtest(target, source, env):
#Assuming I have a CMake file that does this in the directory...
subprocess.run(['cmake', '../'], cwd='build/')
subprocess.run(['make'], cwd='build/')
env2.Command(['gtest.h', 'build/libgtest.a', 'build/libgtest_main.a'], [], buildGtest)
SConscript for my unit test:
env2 = env.Clone()
env2.Require('{path to gtest}/build/include/gtest.h') # This doesn't seem to work
env2.Append(CPPPATH='{path to gtest}/build/include')
env2.Object(target = 'test1.o', source = 'test1.cpp')
You need to ensure that your CPPPATH includes the location where googletest generates the header file.
This is not great. You should specify at least one source file. I'm guessing you have a CmakeList.txt and a bunch of source files you should list here, so it will rebuild if they change.
env2.Command(['gtest.h', 'build/libgtest.a', 'build/libgtest_main.a'], [], buildGtest)
Additionally I'd change your buildGtest to the following.
buildGtest = ["cd build; cmake ../','cd build; make']
And get rid of your function.
env2.Command(['gtest.h', 'build/libgtest.a', 'build/libgtest_main.a'], [], buildGtest)
Then SCons will scan your source file and know that it requires that file.
You can check the dependency tree that SCons is aware of by running as
scons --tree=prune
It may be quite verbose, but you'll see all the dependencies.
You should also be able to run as
scons --tree=prune {path to my unit test}/test.o
When compiling a SConscript using SConscript("src/SConscript", variant_dir=stage_dir) all files encountered in the SConscript file is copied to the stage_dir. This is the behaviour I want.
But with the current setup of the compiler we end up with error messages that is like this:
g++ -o build\mingw_d\main.o
build\mingw_d\main.cpp: In function 'bool some_function()':
build\mingw_d\main.cpp:42:1: error: Some error message.
This points to the copied file (quite naturally since that is the file that is passed into the compiler).
This causes however our IDEs and editors to point to the wrong file to edit, as we want to edit the file in the #/src/ folder.
I know there is a duplicate=0 setting in the SConscript tool, but I believe that this is more of a workaround. Since duplicate=1 is the default, I believe there is a practical way to achieve this.
I am trying to get SCons to call uglify on a series of JavaScript files, for continuous integration reasons...
However even though the 'duplicate=False' flag is passed to 'env.SConscript' calls; to prevent scons from copying sources to the build output folder. It flags files with the same source and target extension as a dependency cycle. Take for example the following simplified project structure:
SConstruct
applications/SConscript
applications/frontend/SConscript
applications/frontend/scripts/app.js
-- SContruct --
UglifyBuilder = env.Builder(
action = 'uglify --source $SOURCE --output $TARGET',
suffix = '.js',
single_source = True)
env['BUILDERS']['Uglify'] = UglifyBuilder
env.SConscript('applictions/SConscript', 'env',
variant_dir=os.path.join('build', 'debug', 'applications'),
duplicate=False)
-- applications/SConscript --
env.SConscript('frontend/SConscript', 'env', duplicate=False)
-- applications/frontend/SConscript --
env.Uglify('scripts/app.js')
When you run SCons on this (clean or not) the following output is generated; Stating that there is a dependency cycle between the target files:
$ scons applications/frontend/
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: building associated VariantDir targets: build/debug/applications/frontend
scons: `applications/frontend' is up to date.
scons: done building targets.
scons: *** Found dependency cycle(s):
build/debug/applications/frontend/scripts/app.js -> build/debug/applications/frontend/scripts/app.js
I also tried to pass an absolute path to the source file, but unfortunately with the same result. And sure I could change the output suffix to '.min.js', but there are other cases to come up with that output files with the same leader-path/filename as some arbitrary input source.
Am I misunderstanding SCons here? Or can somebody point me in the right direction of solving this?
Because if this is one of those cases where 'it isn't a bug it's a feature' applies I am going to pull my hair out. :)
Thanks!
I'm really sorry for you and your hair, but this behaviour doesn't have anything to do with variant_dir, and the "duplicate" option won't help you out either.
What your example boils down to is:
You have a file "app.js".
You have an Uglify Builder, mapping *.js files to *.js files.
You tell this Builder to do his stuff with "app.js".
Of course this leads to a dependency cycle, because your source and target have the same filename. Note, that SCons detects the cycle from build/debug/.../app.js to itself...and not to applications/frontend/scripts/app.js.
For breaking this cycle, you have to either rename one of your suffixes (*.in for source files?), or the file stem, e.g. you could name your source file "app_pretty.js" and then call
env.Uglify('scripts/app.js', 'scripts/app_pretty.js')
I'm trying to make SCons check out a git repo that I need (and hopefully keep that repo up-to-date). The problem is that I have to tell it which files the git repo contains for it to use them in the build, and if I do that, SCons will create the repo before it tries to clone it.
For example, say I want to clone GStreamer, and use the create-uninstalled-setup.sh script (this isn't what I'm actually doing, but it's a much simpler/faster script that exibits the same problem):
Command(['gstreamer/.git', 'gstreamer/scripts/create-uninstalled-setup.sh'],
None, 'git clone git://anongit.freedesktop.org/git/gstreamer/gstreamer')
Command('~/gst/git', 'gstreamer/scripts/create-uninstalled-setup.sh',
'gstreamer/scripts/create-uninstalled-setup.sh')
But it fails because SCons creates gstreamer/scripts before it tries to clone gstreamer:
$ scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
git clone git://anongit.freedesktop.org/git/gstreamer/gstreamer
fatal: destination path 'gstreamer' already exists and is not an empty directory.
scons: *** [gstreamer/.git] Error 128
scons: building terminated because of errors.
$ ls gstreamer/
scripts
If I tell the first command that it creates the "gstreamer" folder, it creates a dependency cycle:
Command(['gstreamer', 'gstreamer/scripts/create-uninstalled-setup.sh'],
None, 'git clone git://anongit.freedesktop.org/git/gstreamer/gstreamer')
Command('~/gst/git', 'gstreamer/scripts/create-uninstalled-setup.sh',
'gstreamer/scripts/create-uninstalled-setup.sh')
$ scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: done building targets.
scons: *** Found dependency cycle(s):
gstreamer/scripts -> gstreamer/scripts/create-uninstalled-setup.sh -> gstreamer/scripts
gstreamer/scripts/create-uninstalled-setup.sh -> gstreamer/scripts -> gstreamer/scripts> /create-uninstalled-setup.sh
File "/usr/lib/scons/SCons/Taskmaster.py", line 1019, in cleanup
If I don't tell the first command that it creates "create-uninstalled-setup.sh", it gets confused because it doesn't exist:
Command(['gstreamer'],
None, 'git clone git://anongit.freedesktop.org/git/gstreamer/gstreamer')
Command('~/gst/git', 'gstreamer/scripts/create-uninstalled-setup.sh',
'gstreamer/scripts/create-uninstalled-setup.sh')
$ scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: *** [~/gst/git] Source `gstreamer/scripts/create-uninstalled-setup.sh' not found, needed by target `~/gst/git'.
scons: building terminated because of errors.
As a workaround, I can rm -rf the folder before I clone, but that's obviously not ideal:
Command(['gstreamer/.git', 'gstreamer/scripts/create-uninstalled-setup.sh'], None,
'rm -rf gstreamer && git clone git://anongit.freedesktop.org/git/gstreamer/gstreamer')
Command('~/gst/git', 'gstreamer/scripts/create-uninstalled-setup.sh',
'gstreamer/scripts/create-uninstalled-setup.sh')
Is there a better way of handling this?
Since there is no way to tell git that the dir is indeed empty (even though it has empty subdirs), and we cant figure out how to tell SCons not to auto-create the subdirs, you could create your own dependency check and call git clone with the SCons Execute() function, which is executed before any SCons builtin dependency checking is performed, as follows:
import os.path
setup_file = 'gstreamer/scripts/create-uninstalled-setup.sh'
if not os.path.exists(setup_file)
Execute('git clone git://anongit.freedesktop.org/git/gstreamer/gstreamer')
Command('~/gst/git', setup_file, setup_file)
What happens and what seem to be the problem is that during preparing to build a target all the directory up to it is created if they don't exist. This seem to be done by building the directory node which by default is done by just creating the directory.
This means that your first try will create the gstreamer directory by simply creating the directory. The reason your second try fails I really don't know. And the third is because you simply has no rule to build gstreamer/scripts/create-uninstalled-setup.sh.
A solution seem to be to define a rule that builds the gstreamer directory:
Command(Dir("gstreamer"), [], "git clone git://anongit.freedesktop.org/git/gstreamer/gstreamer")
then a rule to create gstreamer/scripts/create-uninstalled-setup.sh. Ideally I'd like to include it in the above rule, but it doesn't seem to work. However since it will be created when the gstreamer directory is being created by the above rule you only need a dummy rule (no dependencies, no further action - the preparation to build it will do the work):
Command("gstreamer/scripts/create-uninstalled-setup.sh", [], [])
The drawback with this approach is that it would require every source found in the git would have to be explicitely tell scons about these, at least I've found no way to do this automatically. If this is a problem to you you would have to make sure you clone the git when the SConstruct file is being read (ie use the Execute function as in Brady's answer).
Suppose I have an SConstruct file that looks like this:
env = Environment()
env.Program("a", ["a.c", "util.c"])
env.Program("b", ["b.c", "util.c"])
This build works properly with no SCons warning messages. However, if I modify this to specify different libraries for each Program build (the actual libraries are not relevant):
env.Program("a", ["a.c", "util.c"], LIBS="m")
env.Program("b", ["b.c", "util.c"], LIBS="c")
then I get the warning:
scons: warning: Two different environments were specified for target util.o,
but they appear to have the same action: $CC -o $TARGET -c $CFLAGS $CCFLAGS $_CCCOMCOM $SOURCES
This appears to be caused by the Program builder automatically creating a new environment for building the sources, even though it is just the LIBS variable that is different (and so only the link step needs to have a different environment). I can work around this by doing something like:
util = env.Object("util.c")
env.Program("a", ["a.c"] + util, LIBS="m")
env.Program("b", ["b.c"] + util, LIBS="c")
This uses a single Object builder for building util.c, then using the precompiled object file in each Program build, thus avoiding the warning. However, this should not really be necessary. Is there a more elegant way to work around this problem? Or is this actually a bug in SCons that should be fixed?
Context: I have nearly 2000 C source files compiled into about 20 libraries and 120 executables with lots of shared sources. I created the SConstruct file from the previous proprietary build system using a conversion script I wrote. There are about 450 "Two different environments" warning messages produced by SCons for a full build using my current SConstruct.
I found a workaround that doesn't involve creating extra variables to hold the object file nodes:
env.Program("a", ["a.c", env.Object("util.c")], LIBS="m")
env.Program("b", ["b.c", env.Object("util.c")], LIBS="c")
This isolates the build of util.c within a single environment. Although it is specified twice, once for each Program, SCons doesn't warn about this because it's the same source built with the same env object. Of course SCons only compiles the source once in this case.
You may use the Split function and a custom helper to simplify the build process for large projects:
def create_objs(SRCS, path=""):
return [env.Object(path+src+".cpp") for src in SRCS]
prg1 = Split("file_1 file_2 file_N")
prg2 = Split("file_2 file_5 file_8")
env.Program("a", create_objs(prg1), LIBS="x")
env.Program("b", create_objs(prg2), LIBS="y")
The object files are created only once, and they can be used in multiple builds. Hope this helps...
One issue I found in my code was that I was not using the target object path correctly. Or in otherwords I had a variant dir directive, but instead of using BUILDPATH i ended up using my original source code path. This way Scons was finding the object generated in target BUILDPATH and source path.
Creating a static library out of the first set of files and linking the library to the next set of files (which have some files in common with the first set) to create a target works as well.
env.StaticLibrary ("a", ["a.c","util.c"], LIBS = "m")
env.Program ("b", ["b.c","util.c"], LIBS = ["c","a"])