I'm trying to make SCons check out a git repo that I need (and hopefully keep that repo up-to-date). The problem is that I have to tell it which files the git repo contains for it to use them in the build, and if I do that, SCons will create the repo before it tries to clone it.
For example, say I want to clone GStreamer, and use the create-uninstalled-setup.sh script (this isn't what I'm actually doing, but it's a much simpler/faster script that exibits the same problem):
Command(['gstreamer/.git', 'gstreamer/scripts/create-uninstalled-setup.sh'],
None, 'git clone git://anongit.freedesktop.org/git/gstreamer/gstreamer')
Command('~/gst/git', 'gstreamer/scripts/create-uninstalled-setup.sh',
'gstreamer/scripts/create-uninstalled-setup.sh')
But it fails because SCons creates gstreamer/scripts before it tries to clone gstreamer:
$ scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
git clone git://anongit.freedesktop.org/git/gstreamer/gstreamer
fatal: destination path 'gstreamer' already exists and is not an empty directory.
scons: *** [gstreamer/.git] Error 128
scons: building terminated because of errors.
$ ls gstreamer/
scripts
If I tell the first command that it creates the "gstreamer" folder, it creates a dependency cycle:
Command(['gstreamer', 'gstreamer/scripts/create-uninstalled-setup.sh'],
None, 'git clone git://anongit.freedesktop.org/git/gstreamer/gstreamer')
Command('~/gst/git', 'gstreamer/scripts/create-uninstalled-setup.sh',
'gstreamer/scripts/create-uninstalled-setup.sh')
$ scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: done building targets.
scons: *** Found dependency cycle(s):
gstreamer/scripts -> gstreamer/scripts/create-uninstalled-setup.sh -> gstreamer/scripts
gstreamer/scripts/create-uninstalled-setup.sh -> gstreamer/scripts -> gstreamer/scripts> /create-uninstalled-setup.sh
File "/usr/lib/scons/SCons/Taskmaster.py", line 1019, in cleanup
If I don't tell the first command that it creates "create-uninstalled-setup.sh", it gets confused because it doesn't exist:
Command(['gstreamer'],
None, 'git clone git://anongit.freedesktop.org/git/gstreamer/gstreamer')
Command('~/gst/git', 'gstreamer/scripts/create-uninstalled-setup.sh',
'gstreamer/scripts/create-uninstalled-setup.sh')
$ scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: *** [~/gst/git] Source `gstreamer/scripts/create-uninstalled-setup.sh' not found, needed by target `~/gst/git'.
scons: building terminated because of errors.
As a workaround, I can rm -rf the folder before I clone, but that's obviously not ideal:
Command(['gstreamer/.git', 'gstreamer/scripts/create-uninstalled-setup.sh'], None,
'rm -rf gstreamer && git clone git://anongit.freedesktop.org/git/gstreamer/gstreamer')
Command('~/gst/git', 'gstreamer/scripts/create-uninstalled-setup.sh',
'gstreamer/scripts/create-uninstalled-setup.sh')
Is there a better way of handling this?
Since there is no way to tell git that the dir is indeed empty (even though it has empty subdirs), and we cant figure out how to tell SCons not to auto-create the subdirs, you could create your own dependency check and call git clone with the SCons Execute() function, which is executed before any SCons builtin dependency checking is performed, as follows:
import os.path
setup_file = 'gstreamer/scripts/create-uninstalled-setup.sh'
if not os.path.exists(setup_file)
Execute('git clone git://anongit.freedesktop.org/git/gstreamer/gstreamer')
Command('~/gst/git', setup_file, setup_file)
What happens and what seem to be the problem is that during preparing to build a target all the directory up to it is created if they don't exist. This seem to be done by building the directory node which by default is done by just creating the directory.
This means that your first try will create the gstreamer directory by simply creating the directory. The reason your second try fails I really don't know. And the third is because you simply has no rule to build gstreamer/scripts/create-uninstalled-setup.sh.
A solution seem to be to define a rule that builds the gstreamer directory:
Command(Dir("gstreamer"), [], "git clone git://anongit.freedesktop.org/git/gstreamer/gstreamer")
then a rule to create gstreamer/scripts/create-uninstalled-setup.sh. Ideally I'd like to include it in the above rule, but it doesn't seem to work. However since it will be created when the gstreamer directory is being created by the above rule you only need a dummy rule (no dependencies, no further action - the preparation to build it will do the work):
Command("gstreamer/scripts/create-uninstalled-setup.sh", [], [])
The drawback with this approach is that it would require every source found in the git would have to be explicitely tell scons about these, at least I've found no way to do this automatically. If this is a problem to you you would have to make sure you clone the git when the SConstruct file is being read (ie use the Execute function as in Brady's answer).
Related
I'm working on adding googletest unit testing to a large project.
We have a top level SConstruct, which calls SConscripts for each of the git submodules, which may further call other SConstructs contained in each component.
One of these SConscripts will build the googletest framework, which will create the 'gtest.h' file among others. I need to include 'gtest.h' in my source file (test1.cpp) which is in a different repo that has it's own SConscript.
How can I require the gtest.h in test SConscript file before test1.cpp consumes it?
Imagine something like:
Top level SConstruct:
env.SConscript('{path to Gtest}/SConscript')
env.SConscript('{path to my unit test}/SConscript')
SConscript for building Gtest:
env2 = env.Clone()
def buildGtest(target, source, env):
#Assuming I have a CMake file that does this in the directory...
subprocess.run(['cmake', '../'], cwd='build/')
subprocess.run(['make'], cwd='build/')
env2.Command(['gtest.h', 'build/libgtest.a', 'build/libgtest_main.a'], [], buildGtest)
SConscript for my unit test:
env2 = env.Clone()
env2.Require('{path to gtest}/build/include/gtest.h') # This doesn't seem to work
env2.Append(CPPPATH='{path to gtest}/build/include')
env2.Object(target = 'test1.o', source = 'test1.cpp')
You need to ensure that your CPPPATH includes the location where googletest generates the header file.
This is not great. You should specify at least one source file. I'm guessing you have a CmakeList.txt and a bunch of source files you should list here, so it will rebuild if they change.
env2.Command(['gtest.h', 'build/libgtest.a', 'build/libgtest_main.a'], [], buildGtest)
Additionally I'd change your buildGtest to the following.
buildGtest = ["cd build; cmake ../','cd build; make']
And get rid of your function.
env2.Command(['gtest.h', 'build/libgtest.a', 'build/libgtest_main.a'], [], buildGtest)
Then SCons will scan your source file and know that it requires that file.
You can check the dependency tree that SCons is aware of by running as
scons --tree=prune
It may be quite verbose, but you'll see all the dependencies.
You should also be able to run as
scons --tree=prune {path to my unit test}/test.o
I don't have much experience with makefiles, but I am trying to compile and run the program "hdf5ToMds.c" found on this Github repository: https://github.com/MDSplus/mdsplus/tree/alpha/hdf5
My steps towards using the Makefile.in in the folder were as follows:
autoscan
mv configure.scan configure.ac
aclocal
autoheader
autoconf
./configure
make
Now when I run the make I get the error:
Makefile:1: Makefile.inc: No such file or directory
make: *** No rule to make target 'Makefile.inc'. Stop.
Am I missing something obvious? I'm not very familiar with this whole process of getting the code from Github and having to make it. I do have the entire repository cloned, just in case that's relevant.
I have a scons project with many SConscript files in the subdirectories. In one of those SConscript files I want to check if particular library is installed on the build host. Here is code snippet:
Import("env")
conf = Configure(env)
if conf.CheckLibWithHeader(...):
doSomething()
env.Library(...)
When I execute build scons fails with strnge error without error message like this:
scons: ***
File "/path/to/SConscript", line 3, in <module>
Line 3 is where I call Configure(). I guess I am doing something not allowed with Configure() function but I cannot find any explanation in scons documentation.
Please help me to debug this.
So the problem was in one of the sibling SConscript files. There was a call to Configure(env) but configuration object was not disposed with conf.Finish() call.
This was an issue because SCons does not allow to create more than one configuration object at any moment. I found this rule by analysing SCons sources - I don't know if this is documented.
Thus when I made my call to Configure(env) SCons thrown an exception because it seen there is another active configuration object orphaned by sibling SConscript.
I am trying to get SCons to call uglify on a series of JavaScript files, for continuous integration reasons...
However even though the 'duplicate=False' flag is passed to 'env.SConscript' calls; to prevent scons from copying sources to the build output folder. It flags files with the same source and target extension as a dependency cycle. Take for example the following simplified project structure:
SConstruct
applications/SConscript
applications/frontend/SConscript
applications/frontend/scripts/app.js
-- SContruct --
UglifyBuilder = env.Builder(
action = 'uglify --source $SOURCE --output $TARGET',
suffix = '.js',
single_source = True)
env['BUILDERS']['Uglify'] = UglifyBuilder
env.SConscript('applictions/SConscript', 'env',
variant_dir=os.path.join('build', 'debug', 'applications'),
duplicate=False)
-- applications/SConscript --
env.SConscript('frontend/SConscript', 'env', duplicate=False)
-- applications/frontend/SConscript --
env.Uglify('scripts/app.js')
When you run SCons on this (clean or not) the following output is generated; Stating that there is a dependency cycle between the target files:
$ scons applications/frontend/
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: building associated VariantDir targets: build/debug/applications/frontend
scons: `applications/frontend' is up to date.
scons: done building targets.
scons: *** Found dependency cycle(s):
build/debug/applications/frontend/scripts/app.js -> build/debug/applications/frontend/scripts/app.js
I also tried to pass an absolute path to the source file, but unfortunately with the same result. And sure I could change the output suffix to '.min.js', but there are other cases to come up with that output files with the same leader-path/filename as some arbitrary input source.
Am I misunderstanding SCons here? Or can somebody point me in the right direction of solving this?
Because if this is one of those cases where 'it isn't a bug it's a feature' applies I am going to pull my hair out. :)
Thanks!
I'm really sorry for you and your hair, but this behaviour doesn't have anything to do with variant_dir, and the "duplicate" option won't help you out either.
What your example boils down to is:
You have a file "app.js".
You have an Uglify Builder, mapping *.js files to *.js files.
You tell this Builder to do his stuff with "app.js".
Of course this leads to a dependency cycle, because your source and target have the same filename. Note, that SCons detects the cycle from build/debug/.../app.js to itself...and not to applications/frontend/scripts/app.js.
For breaking this cycle, you have to either rename one of your suffixes (*.in for source files?), or the file stem, e.g. you could name your source file "app_pretty.js" and then call
env.Uglify('scripts/app.js', 'scripts/app_pretty.js')
I would like to build all .c files in a subdirectory. I figured I would do something like this:
src/foo/SConscript contains:
import glob;
here = Dir('.');
sourcefiles_raw = glob.glob(here.path+'/*.c');
print(sourcefiles_raw);
# print them for debugging
# ... then build them (in the process, making scons aware of dependencies)
src/SConscript contains:
SConscript(['foo/SConscript']);
SConstruct contains:
SConscript(['src/SConscript'],build_dir='build');
But it prints [], since glob.glob() runs in the directory build/foo before scons can decide which sourcefiles need to be copied from src/foo to build/foo.
How can I solve this problem?
never mind, it seems you're supposed to (RTFM) and use scons's Glob() rather than glob.glob().