I have a requirement to generate version.cc file from SCons Script. This file should be generated only if any of the source file for a target has changed.
Suppose the SCons script has the following statements
#python function which generates version.cc in the same folder where libtest.a is generated. This will always generate a differnt version.cc because the version string contained inside that will have timestamp
GenerateVersionCode()
#target which uses version.cc
libtest = env.Library('test', ['a.cc', 'b.cc', 'version.cc'])
First time when I run the above code everything is fine. But when I run the same script again, the target 'test' will be rebuilt because of new version.cc which got generated.
My requirement is we should not generate new version of version.cc file if the file is already present and there are no changes in any of the sources (namely a.cc and b.cc in this example)
if not version_file_present:
GenerateVersionCode()
else
if no_changes_in_source:
GenerateVersionCode()
#target which uses version.cc which could be newly generated one or previous one
libtest = env.Library('test', ['a.cc', 'b.cc', 'version.cc'])
One related question on this site suggested something as follows
env.Command(target="version.c", source="version-in.c",
action=PythonFunctionToUpdateContents)
env.Program("foo", ["foo.c", "version.c"])
W.r.to the above suggestion I would want to know the contents of function PythonFunctionToUpdateContents which checks the change in source files since previous build.
As I understand it, you only want to generate version.cc if any of the source files change, and you only want to build the library if version.cc changes or if any of the library source files change. That is, consider version.cc as one of the source files for the library.
If this is the case, you could consider 2 sets of dependencies, both of which would be controlled by the SCons dependency checking.
Its not real clear what the version.cc generation consists of, but lets assume that the python function GenerateVersionCode() will do exactly that: generate version.cc, but wont have any dependency checking related logic.
Here is the SConscript code:
def GenerateVersionCode(env, target, source):
# fill in generation code here
# The version.cc checking
env.Command(target='version.cc',
source=['a.cc', 'b.cc'],
action=GenerateVersionCode)
# The library
env.Library(target='test', source=['version.cc', 'a.cc', 'b.cc'])
It shouldnt be necessary, but this could be taken one step further by explicitly setting a dependency from the Library target to the version.cc target with the SCons Depends() function.
Here is the output I get when I build, and instead of using the GenerateVersionCode() function, I use a simple shell script versionGen.sh, thus changing the call to Command() to this:
env.Command(target='version.cc',
source=['a.cc', 'b.cc'],
action='./versionGen.sh')
Here is the first build:
> scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
g++ -o a.o -c a.cc
g++ -o b.o -c b.cc
./versionGen.sh
g++ -o version.o -c version.cc
ar rc libtest.a version.o a.o b.o
ranlib libtest.a
scons: done building targets.
Then, without having changed anything, I build again, and it does nothing:
> scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: `.' is up to date.
scons: done building targets.
Then, I modify a.cc, and build again, and it generates a new version of version.cc:
> vi a.cc
> scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
g++ -o a.o -c a.cc
./versionGen.sh
g++ -o version.o -c version.cc
ar rc libtest.a version.o a.o b.o
ranlib libtest.a
scons: done building targets.
Related
I am trying to generate LLVM bytecode file from a C source file (hello.c) using CMake. And below is my CMakeLists file.
###### CMakelists.txt ############
cmake_minimum_required(VERSION 2.8.9)
set(CMAKE_C_COMPILER "clang")
set(CMAKE_C_FLAGS "-emit-llvm")
project (hello)
add_executable(hello hello.c)
I am new to CMake and not sure if this is the right way. I could not find any rules to make *.bc in the generated MakeFile
. Please correct me here. I also tried "-save-temps"
Considering this for a single .c file. It would be really helpful if you could give me some hints on generating the same for a complete C project.
I think what you ultimately want is to be able to build a C-program
project with CMake and clang in which source files are compiled to LLVM bitcode
and the executable is linked from the bitcode files.
With CMake, asking clang to to link bitcode files means asking it to link in LTO mode,
with the -flto linkage option.
And you can get clang to compile to LLVM bitcode with the -flto compilation
option, or with the -emit-llvm option.
For illustration here is a Hello World project comprising two source files and one header:
$ ls -R
.:
CMakeLists.txt hello.c hello.h main.c
Here is the:
CMakeLists.txt
cmake_minimum_required(VERSION 3.0.2)
project (hello)
set(CMAKE_C_COMPILER clang)
set(CMAKE_EXE_LINKER_FLAGS ${CMAKE_EXE_LINKER_FLAGS} "-flto")
add_executable(hello main.c hello.c)
target_compile_options(hello PUBLIC ${CMAKE_C_FLAGS} -flto)
#target_compile_options(hello PUBLIC ${CMAKE_C_FLAGS} -emit-llvm)
It will work equally well with:
#target_compile_options(hello PUBLIC ${CMAKE_C_FLAGS} -flto)
target_compile_options(hello PUBLIC ${CMAKE_C_FLAGS} -emit-llvm)
Make a build directory for CMake and go there:
$ mkdir build
$ cd build
Generate the build system:
$ cmake ..
Build:
$ make
Scanning dependencies of target hello
[ 33%] Building C object CMakeFiles/hello.dir/main.c.o
[ 66%] Building C object CMakeFiles/hello.dir/hello.c.o
[100%] Linking C executable hello
[100%] Built target hello
You will not find any *.bc targets in the Makefiles, nor any *.bc files
generated:
$ egrep -r '.*\.bc'; echo Done
Done
$ find -name '*.bc'; echo Done
Done
because the compilation option -flto or -emit-llvm results in an output
file:
CMakeFiles/hello.dir/main.c.o
CMakeFiles/hello.dir/hello.c.o
that adheres to the usual CMake naming convention but is in fact not an object file
but an LLVM bitcode file, as you see:
$ file $(find -name '*.o')
./CMakeFiles/hello.dir/hello.c.o: LLVM IR bitcode
./CMakeFiles/hello.dir/main.c.o: LLVM IR bitcode
The program does the usual thing:
$ ./hello
Hello World!
Later
When I try " make hello.o " it should generate the object file right?
the cmd executes successfully but, could not find the generated object file. Am I doing it right?
You are doing it in one way that is right, though not the only way that is right, but
your expectations are wrong. Look again at:
$ file $(find -name '*.o')
./CMakeFiles/hello.dir/hello.c.o: LLVM IR bitcode
./CMakeFiles/hello.dir/main.c.o: LLVM IR bitcode
You can see there that the .o files that are made from hello.c and main.c
by the CMake-generated makefile are not called hello.o and main.o but hello.c.o
and main.c.o. CMake prefers a compiled filename to preserve the extension of the
source file, and append .o. That is a fairly common practice. So if you wanted
to use the makefile to compile hello.c, the most obviously right way would be
make hello.c.o.
Let's see what actually happens. In my CMake build directory:
$ make VERBOSE=1 hello.c.o
make -f CMakeFiles/hello.dir/build.make CMakeFiles/hello.dir/hello.c.o
make[1]: Entering directory '/home/imk/develop/so/scrap/build'
make[1]: 'CMakeFiles/hello.dir/hello.c.o' is up to date.
make[1]: Leaving directory '/home/imk/develop/so/scrap/build'
There was nothing to be done, because my hello.c.o was up to date. So I'll
delete it and repeat:
$ rm CMakeFiles/hello.dir/hello.c.o
$ make VERBOSE=1 hello.c.o
make -f CMakeFiles/hello.dir/build.make CMakeFiles/hello.dir/hello.c.o
make[1]: Entering directory '/home/imk/develop/so/scrap/build'
Building C object CMakeFiles/hello.dir/hello.c.o
clang -flto -o CMakeFiles/hello.dir/hello.c.o -c /home/imk/develop/so/scrap/hello.c
make[1]: Leaving directory '/home/imk/develop/so/scrap/build'
Now it has been recompiled.
However, because many people - like you - would expect hello.o to be compiled
from hello.c, CMake helpfully defines hello.o as a .PHONY target
that depends on hello.c.o:
$ egrep -A3 'hello.o.*:.*hello.c.o' Makefile
hello.o: hello.c.o
.PHONY : hello.o
So in fact I can do:
$ rm CMakeFiles/hello.dir/hello.c.o
$ make VERBOSE=1 hello.o
make -f CMakeFiles/hello.dir/build.make CMakeFiles/hello.dir/hello.c.o
make[1]: Entering directory '/home/imk/develop/so/scrap/build'
Building C object CMakeFiles/hello.dir/hello.c.o
clang -flto -o CMakeFiles/hello.dir/hello.c.o -c /home/imk/develop/so/scrap/hello.c
make[1]: Leaving directory '/home/imk/develop/so/scrap/build'
make hello.o is another way of making hello.c.o
The problem is that using the -emit-llvm flag does not produce a final binary and stops the configuration tests that CMake performs once that flag is used in them.
Apart from what's already been written about using the LTO infrastructure, you have 3 (or 2 and a half) other alternatives.
One is to use Whole-Program LLVM and use the commands provided to extract the relevant bitcode parts.
The other is to go the manual way of setting up custom targets (see add_custom_target and add_custom_command) on your CMake binary targets, that will get triggered on changes and will reproduce the desired outcome as if executed manually on the command line each time.
Now, on this last point, I had a similar need so I created a CMake project that provides that functionality (llvm-ir-cmake-utils), but allows you to hook up those custom targets on existing ones as you please and see fit without having to rewrite everything from scratch each time.
There are examples in the repo, but in short, it allows you to attach custom targets on already existing CMake targets, e.g.
[...]
add_executable(qux ${SOURCES})
[...]
# this will create a bitcode generating target
# and allow it to depend on the initial target in order to detect source code changes
llvmir_attach_bc_target(qux_bc qux)
add_dependencies(qux_bc qux)
[...]
After make,
$>file CMakeFiles/hello.dir/hello.c.o
CMakeFiles/hello.dir/hello.c.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
if
set(CMAKE_C_FLAGS "-emit-llvm")
written before
project (hello)
In order to obtain IR bitcode, I wrote:
###### CMakelists.txt ############
cmake_minimum_required(VERSION 2.8.9)
project (hello)
set(CMAKE_C_COMPILER "clang")
set(CMAKE_C_FLAGS "-flto")
set(CMAKE_EXE_LINKER_FLAGS ${CMAKE_EXE_LINKER_FLAGS} "-flto")
add_executable(hello hello.c)
target_compile_options(hello PUBLIC ${CMAKE_C_FLAGS} -flto)
I worked several hours in order to have a Makefile working to compile from IR
code to native using lld, then with cmake it was much more faster.
Then reading at cmake generated Makefile, I was able to correct my Makefile:
clang -flto -flto <hello.c.o> ..
this worked but I do not know why -flto is written twice.
Thanl you very much for this post, showing clang as the centralized front end to various llvm provided commands.
I'm trying to use scons functions to detect if a specific header file is available. Given a file main.cpp with a #include "fns.h". I've created this:
env = Environment(CXXFLAGS='-I.')
conf = Configure(env)
if not conf.CheckCXXHeader('fns.h'):
print 'please get fns.h'
Exit(1)
env.Program('main.cpp')
The first time I run it, it actually checks for the header, but after that it always uses the cached result
$ scons
scons: Reading SConscript files ...
Checking for C++ header file fns.h... no
please get fns.h
$ touch fns.h
$ scons
scons: Reading SConscript files ...
Checking for C++ header file fns.h... yes
scons: done reading SConscript files.
scons: Building targets ...
g++ -o main.o -c -I. main.cpp
g++ -o main main.o
scons: done building targets.
$ rm fns.h # delete it
$ scons # should check again here, but doesn't
scons: Reading SConscript files ...
Checking for C++ header file fns.h... (cached) yes
scons: done reading SConscript files.
scons: Building targets ...
g++ -o main.o -c -I. main.cpp
main.cpp:1:17: fatal error: fns.h: No such file or directory
#include "fns.h"
Please check the SCons manual (man scons), the command-line option you're looking for is "--config=force". You can also make it the default behavior for your build by setting the config option in an SConscript file:
SetOption('config', 'force')
I want to do some post-processing on all .o files (e.g. those generated by the Object or StaticObject Builders), no matter what Builder is used by the user.
I'm trying to "override" or "hook" the base environment's builders like so, but I can't come up with anything that works:
old = env['BUILDERS']['StaticObject']
env['BUILDERS']['StaticObject'] = something_that_calls(old)
Is there a prescribed way to hook or override something like the StaticObject builder?
I've seen the question about a Nested SCons Builder, but it doesn't tell me how to replace an existing builder, only supplementing its behavior.
I don't know if there is a blessed way to replace a Builder, but I think you're on the right track. The following (admittedly trivial) example works for me:
def wrap_builder(bld):
return Builder(action = [bld.action, 'echo $TARGET'],
suffix = bld.suffix,
src_suffix = bld.src_suffix)
obj_bld = env['BUILDERS']['Object']
env['BUILDERS']['Object'] = wrap_builder(obj_bld)
env.Program('test', ['test.c'])
with output:
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
gcc -o test.o -c -std=c99 test.c
echo test.o
test.o
gcc -o test test.o
scons: done building targets.
As you can see, the additional (echo) action is performed after building the object file.
I use to use this simple Makefile to compile all .cc files in the current directory
SRCS:=$(wildcard *.cc)
OBJS:=$(SRCS:.cc=)
CXX := clang++
CXXFLAGS := -std=c++11 -g
all: $(OBJS)
I'm trying to translate this into an SConstruct file. I can use scons' Glob built-in to get the .cc file list but I don't know how to remove their suffix (like the OBJS := $(SRCS:.cc=) do). Of course I can write Python code to do the modification but does scons has built-in support for this kind of modification?
UPDATE:
My original SConstruct file (literally list all the .cc files)
env = Environment(CXX="clang++", CXXFLAGS=['-std=c++11', '-g'])
env.Program("1.1.cc")
env.Program("1.2.cc")
env.Program("1.3.cc")
env.Program("1.4.cc")
One version that works
import glob
env = Environment(CXX="clang++", CXXFLAGS=['-std=c++11', '-g'])
sources = glob.glob("./*.cc")
for s in sources:
env.Program(s)
Based on your response to my question in the comments above, it appears that you want SCons to automatically create a binary or object file based on the source file.
This can be done in SCons as follows:
env = Environment()
# This will build example.o
env.Object('example.cc')
# This will build main
env.Program('main.cc')
Here is the output from this build:
$ scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
g++ -o example.o -c example.cc
g++ -o main.o -c main.cc
g++ -o main main.o
scons: done building targets.
If you need to build a binary with more than one source file, then you will need to specify the binary name, as follows:
Program(target = 'myBinary', source = ['main.cc', 'example.cc'])
This will create main.o and example.o, but cant know the name of the binary/program, so you have to specify it.
As for your example with glob, Scons has a built-in Glob() function, so you could do the following:
env = Environment(CXX="clang++", CXXFLAGS=['-std=c++11', '-g'])
sources = Glob("./*.cc")
for s in sources:
env.Program(s)
The SCons Glob() function is not recursive, so if you need to recursively list files, then you'll need to do it differently.
I use a wrapper method to combine static libraries as shown below.
def MergeLibs(env, tgt, src_list)
....
return lib
and used as,
lib = env.MergeLibs(tgt, src_lists)
env.Depends(lib, <path_to_lib1>)
...
env.Depends(lib, <path_to_libn>)
But MergeLibs() method is being executed in scons parse phase itself.
How can I use dependencies here.
Thanks
Well I'm not too sure on the details of the MergeLib step but it seems like you would want something else to depend on your merge lib step.. like your final program?
import SCons
env = Environment()
def merge_libs(self, target, source, env):
print "hi"
return env.StaticLibrary(target, source)
env.Append(BUILDERS = {'MergeLibs' : merge_libs})
lib = env.MergeLibs('mrglibs', ['some_file.cpp', 'some_file2.cpp'], env)
prog = env.Program('test.cpp')
env.Depends(prog, lib)
This gives me the output:
scons: Reading SConscript files ...
hi
scons: done reading SConscript files.
scons: Building targets ...
g++ -o some_file.o -c some_file.cpp
g++ -o some_file2.o -c some_file2.cpp
ar rc libmrglibs.a some_file.o some_file2.o
ranlib libmrglibs.a
g++ -o test.o -c test.cpp
g++ -o test test.o
scons: done building targets.
So its definitely still read during the parse phase (I think it has to be) but it should get you what you want.