scons difference between ccflags and shccflags - scons

What is the reason for having both a CCFLAGS and a SHCCFLAGS variable in the SCons environment?
I am trying to modify a large program build that I did not write myself. When I add the command:
env.Append(CCFLAGS=["-I%s" % amber_dir, "-DAMBER"])
the compiler runs without the flags I added. But when I do
env.Append(SHCCFLAGS=["-I%s" % amber_dir, "-DAMBER"])
the compiler adds my flags as wanted. Somewhere in SCons' innards, CCFLAGS are is not passed to SHCCFLAGS. But why have a CCFLAGS and a SHCCFLAGS to begin with?

This is taken from the SCons User's Guide: SCons Construction Variable
CCFLAGS
General options that are passed to the C and C++ compilers.
CPPFLAGS
User-specified C preprocessor options. These will be included in any
command that uses the C preprocessor, including not just compilation
of C and C++ source files via the $CCCOM, $SHCCCOM, $CXXCOM and
$SHCXXCOM command lines, but also the $FORTRANPPCOM, $SHFORTRANPPCOM,
$F77PPCOM and $SHF77PPCOM command lines used to compile a Fortran
source file, and the $ASPPCOM command line used to assemble an
assembly language source file, after first running each file through
the C preprocessor. Note that this variable does not contain -I (or
similar) include search path options that scons generates
automatically from $CPPPATH. See $_CPPINCFLAGS, below, for the
variable that expands to those options.
SHCCFLAGS
Options that are passed to the C and C++ compilers to generate
shared-library objects.
SHCCCOM
The command line used to compile a C source file to a shared-library
object file. Any options specified in the $SHCFLAGS, $SHCCFLAGS and
$CPPFLAGS construction variables are included on this command line.
The most interesting of the 4 mentioned above are the SHCCCOM and CPPFLAGS variables. Try your test again setting CPPFLAGS instead of CCFLAGS.
A general comment about the flags you are setting: with SCons, its generally better to set include paths with the CPPPATH variable, and to set defines with the CPPDEFINES. When using these variable, you dont need to include the -I, nor the -D flags, SCons will add it for you in a platform-independent manner. Here's an example:
env.Append(CPPPATH=amber_dir)
env.Append(CPPDEFINES='AMBER')
You'll need to test this to see if the SharedObject() and/or SharedLibrary() builders use those, I would imagine they would.
Answers to questions in comment below:
SHCCCOM is just used to see what the command line looks like, sometimes you may need to use it without using the SCons builders directly. SHCCFLAGS is a super-set of CPPFLAGS and others and should be used when you have flags that you only want to pass to SharedLibrary() or SharedObject(). CPPFLAGS applies to both static and shared compilations.

Related

SConstruct 101—moving on from Makefiles

Like
make,
scons has a large number of predefined variables and rules. (Try scons | wc on an SConstruct containing env = Environment(); print(env.Dump()) to see how extended the set is.)
But suppose we aren't after the wizardry of presets but rather want to do something a lot more primitive—simulating launching a few instructions from the (bash, etc) command line?
Also suppose we're quite happy with the default Decider('MD5'). What is the translation of the one-souce-one-target:
out/turquoise.xyz: out/chartreuse.xyz
chartreuse_to_turquoise $< $#
of the two-source-one-target:
out/purple.xyz: out/lilac.xyz out/salmon.xyz
gen_purple $< $#
and of:
run_this:
python prog.py
which we would run on-demand by typing make run_this?
What does the SConstruct for these elementary constructs look like?
All the answers you're looking for are in the users guide (and manpage)
Firstly, assuming you don't want to scan the input files to add included files specified in the input files, you can use Commmand()
(See info here: https://scons.org/doc/production/HTML/scons-user.html#chap-builders-commands)
Then you'll want an alias to specify an a non file command line target
(See here:https://scons.org/doc/production/HTML/scons-user.html#chap-alias)
Putting those two together yields
env=Environment()
# one source, one target
env.Command('out/turquoise.xyz', 'out/chartreuse.xyz', 'chartreuse_to_turquoise $SOURCE $TARGET')
# Two source, one target
env.Command('out/purple.xyz',['out/lilac.xyz','out/salmon.xyz'], 'gen_purple $SOURCES $TARGET')
# And your .phony make target which is actually not great for reproducibility and determining when it should be rerun, because you do not specify any sources or targets
env.Alias('run_this','python prog.py')
Note: SCons doesn't NOT propagate your shell environment variables. So if you depend on (for example) a non system path in your PATH, you'll need to explicitly specify that in env['ENV']['PATH'] for example. For more details take a read through the users guide, manpage and FAQ.
https://scons.org/doc/production/HTML/scons-user.html
https://scons.org/doc/production/HTML/scons-man.html
https://scons.org/faq.html
And you can reach the community directly via our discord server, IRC channel, or users mailing list

Qemu and LD_LIBRARY_PATH variable

When I exec qemu-aarch64 with a binary which is using shared libraries I get the following:
qemu-aarch64 -L /usr/aarch64-linux-gnu ./test
./test: error while loading shared libraries: testlibrary.so.1: cannot open shared object file: No such file or directory
Obviously it is because test does not know where the shared libraries are.
Thus:
qemu-aarch64 -L /usr/aarch64-linux-gnu -E LD_PRELOAD="/home/test/libraries/testlibrary.so.1 /home/test/libraries/testlibrary2.so.1" ./test
hi!
Ok, it works; but when I use LD_LIBRARY_PATH it does not work:
qemu-aarch64 -L /usr/aarch64-linux-gnu -E LD_LIBRARY_PATH="/home/test/libraries ./test
./test: error while loading shared libraries: testlibrary.so.1: cannot open shared object file: No such file or directory
The difference between LD_PRELOAD and LD_LIBRARY_PATH, from ld.so man:
LD_PRELOAD:
A list of additional, user-specified, ELF shared objects
to be loaded before all others. This feature can be used
to selectively override functions in other shared objects.
The items of the list can be separated by spaces or
colons, and there is no support for escaping either
separator. The objects are searched for using the rules
given under DESCRIPTION. Objects are searched for and
added to the link map in the left-to-right order specified
in the list.
In secure-execution mode, preload pathnames containing
slashes are ignored. Furthermore, shared objects are
preloaded only from the standard search directories and
only if they have set-user-ID mode bit enabled (which is
not typical).
Within the names specified in the LD_PRELOAD list, the
dynamic linker understands the tokens $ORIGIN, $LIB, and
$PLATFORM (or the versions using curly braces around the
names) as described above in Dynamic string tokens. (See
also the discussion of quoting under the description of
LD_LIBRARY_PATH.)
There are various methods of specifying libraries to be
preloaded, and these are handled in the following order:
(1) The LD_PRELOAD environment variable.
(2) The --preload command-line option when invoking the
dynamic linker directly.
(3) The /etc/ld.so.preload file (described below).
And,
LD_LIBRARY_PATH:
A list of directories in which to search for ELF libraries
at execution time. The items in the list are separated by
either colons or semicolons, and there is no support for
escaping either separator. A zero-length directory name
indicates the current working directory.
This variable is ignored in secure-execution mode.
Within the pathnames specified in LD_LIBRARY_PATH, the
dynamic linker expands the tokens $ORIGIN, $LIB, and
$PLATFORM (or the versions using curly braces around the
names) as described above in Dynamic string tokens. Thus,
for example, the following would cause a library to be
searched for in either the lib or lib64 subdirectory below
the directory containing the program to be executed:
$ LD_LIBRARY_PATH='$ORIGIN/$LIB' prog
(Note the use of single quotes, which prevent expansion of
$ORIGIN and $LIB as shell variables!)
Why does it work with LD_PRELOAD and not with LD_LIBRARY_PATH?
The library you're opening with LD_PRELOAD is "testlibrary.so.1", but the library that the dynamic loader otherwise looks for is "testlibrary.so.3", which suggests there's a mismatch between the library version you have and the library version the binary is linked against, which maybe the LD_PRELOAD is side-stepping. Does LD_LIBRARY_PATH work if you make sure that you have the file in that directory that the binary is looking for ?

In a Makefile do we use 'prefix' or 'PREFIX'?

Currently in my Makefile I have:
prefix ?= /usr/local
So that I can override prefix value when calling make, like in the following:
make prefix="/new_path"
My question is: does a convention exist for naming this variable inside a Makefile, especially do I have to call it prefix or PREFIX?
This matter since lower or upper case matters in this situation!
Note that I do not use autotools, just a "simple" Makefile
The variables for installation directories section of the GNU make manual discusses prefix.
I believe the autotools use a similarly-cased prefix variable/configure flag.
Compare to DESTDIR however.

Setting RPATH order in QMake

I have a Linux Qt program. I'd like it to preferentially use the (dynamic) Qt libraries in the executable's directory if they exist, otherwise use the system's Qt libs. RPATH to the rescue.
I add this line to the qmake's .pro file:
QMAKE_LFLAGS += '-Wl,-rpath,\'\$$ORIGIN\''
and looking at the resulting executable with readelf I see:
0x000000000000000f (RPATH) Library rpath: [$ORIGIN:/usr/local/Trolltech/Qt-5.2.0/lib]
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN:/usr/local/Trolltech/Qt-5.2.0/lib]
Seems right, but ldd shows it's using the system version:
libQt5Core.so.5 => /usr/local/Trolltech/Qt-5.2.0/lib/libQt5Core.so.5 (0x00007f2d2fe09000)
If I manually edit qmake's resulting Makefile to swap the order of the two rpaths, so $ORIGIN comes after /usr/local/..., I get the right behavior:
0x000000000000000f (RPATH) Library rpath: [/usr/local/Trolltech/Qt-5.2.0/lib:$ORIGIN]
0x000000000000001d (RUNPATH) Library runpath: [/usr/local/Trolltech/Qt-5.2.0/lib:$ORIGIN]
libQt5Core.so.5 => ./libQt5Core.so.5 (0x00007fb92aba9000)
My problem is with how qmake constructs the final LFLAGS variable. I can't figure out how to make it put my addition ($ORIGIN) after the system library. Any ideas?
You can add the following to your .pro file to force the dynamic linker to look in the same directory as your Qt application at runtime in Linux :
unix:{
# suppress the default RPATH if you wish
QMAKE_LFLAGS_RPATH=
# add your own with quoting gyrations to make sure $ORIGIN gets to the command line unexpanded
QMAKE_LFLAGS += "-Wl,-rpath,\'\$$ORIGIN\'"
}
If you want it to look in a subdirectory of the executable path, you can use :
QMAKE_LFLAGS += "-Wl,-rpath,\'\$$ORIGIN/libs\'"
Note that you should have the .so files with the exact same name in your application directory. For example you should copy libQt5Core.so.5.2.0 to your application directory with the name libQt5Core.so.5. Now the ldd shows the directory of the application.
You can also have libQt5Core.so.5.2.0 and a link to it with the name libQt5Core.so.5 in the application directory.
As far as my research can say, you can only add RPATH at the beginning of the list with QMake.
But if you are on Linux and can install chrpath, you can hack your way around that.
Add this block at the end of your .pro file
# Add spacing since chrpath cannot expand RPATH length
QMAKE_RPATHDIR = \
/XYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXY1\
/XYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXY2\
/XYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXY3\
/XYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXY4
QMAKE_POST_LINK += 'chrpath -r \'/my/qt/installation:\$$ORIGIN\' $$OUT_PWD/mybinaryname;'
I'm taking a bit of a guess at what's happening, but it's based on knowing some of the odd behaviours of ld.
check for the presence of an LD_LIBRARY_PATH variable that will come into effect before the processing of a RUNPATH variable. Because of the presence of both RPATH and RUNPATH, the LD_LIBRARY_PATH rule comes into effect, so if it's set then unset it.
Secondly, I'd never expect to see:
libQt5Core.so.5 => ./libQt5Core.so.5 (0x00007fb92aba9000)
in the output of ldd, I would always see the expansion of $ORIGIN to the directory of the binary (maybe you shortened it?), so I would have expected:
libQt5Core.so.5 => /path/to/bin/./libQt5Core.so.5 (0x00007fb92aba9000)
Which means it sounds like the LD_LIBRARY_PATH expansion is .:/usr/local/Trolltech/Qt-5.2.0/lib, which to me sounds like you've got environmental overrides happening.
qmake would always append the QMAKE_RPATHDIR with the QT_INSTALL_LIBS internally defined in $(QT_DIR)/mkspecs/features/qt.prf file:
170: relative_qt_rpath:!isEmpty(QMAKE_REL_RPATH_BASE):contains(INSTALLS, target):\
173: QMAKE_RPATHDIR += $$relative_path($$[QT_INSTALL_LIBS], $$qtRelativeRPathBase())
175: QMAKE_RPATHDIR += $$[QT_INSTALL_LIBS/dev]
179:!isEmpty(QMAKE_LFLAGS_RPATHLINK):!contains(QT_CONFIG, static) {
189: QMAKE_RPATHLINKDIR *= $$unique(rpaths)
So to avoid your application using the QT library from system path, comment out the lines above which append the QMAKE_RPATHDIR and add QMAKE_RPATHDIR=$ORIGIN into your .pro file.

g++ searches /lib/../lib/, then /lib/

According to g++ -print-search-dirs my C++ compiler is searching for libraries in many directories, including ...
/lib/../lib/:
/usr/lib/../lib/:
/lib/:
/usr/lib/
Naively, /lib/../lib/ would appear to be the same directory as /lib/ — lib's parent will have a child named lib, "that man's father's son is my father's son's son" and all that. The same holds for /usr/lib/../lib/ and /usr/lib/
Is there some reason, perhaps having to do with symbolic links, that g++ ought to be configured to search both /lib/../lib/ and /lib/?
If this is unnecessary redundancy, how would one go about fixing it?
If it matters, this was observed on an unmodified install of Ubuntu 9.04.
Edit: More information.
The results are from executing g++ -print-search-dirs with no other switches, from a bash shell.
Neither LIBRARY_PATH nor LPATH are output from printenv, and both echo $LPATH and echo LIBRARY_PATH return blank lines.
An attempt at an answer (which I gathered from a few minutes of looking at the gcc.c driver source and the Makefile environment).
These paths are constructed in runtime from:
GCC exec prefix (see GCC documentation on GCC_EXEC_PREFIX)
The $LIBRARY_PATH environment variable
The $LPATH environment variable (which is treated like $LIBRARY_PATH)
Any values passed to -B command-line switch
Standard executable prefixes (as specified during compilation time)
Tooldir prefix
The last one (tooldir prefix) is usually defined to be a relative path:
From gcc's Makefile.in
# Directory in which the compiler finds libraries etc.
libsubdir = $(libdir)/gcc/$(target_noncanonical)/$(version)
# Directory in which the compiler finds executables
libexecsubdir = $(libexecdir)/gcc/$(target_noncanonical)/$(version)
# Used to produce a relative $(gcc_tooldir) in gcc.o
unlibsubdir = ../../..
....
# These go as compilation flags, so they define the tooldir base prefix
# as ../../../../, and the one of the library search prefixes as ../../../
# These get PREFIX appended, and then machine for which gcc is built
# i.e i484-linux-gnu, to get something like:
# /usr/lib/gcc/i486-linux-gnu/4.2.3/../../../../i486-linux-gnu/lib/../lib/
DRIVER_DEFINES = \
-DSTANDARD_STARTFILE_PREFIX=\"$(unlibsubdir)/\" \
-DTOOLDIR_BASE_PREFIX=\"$(unlibsubdir)/../\" \
However, these are for compiler-version specific paths. Your examples are likely affected by the environment variables that I've listed above (LIBRARY_PATH, LPATH)
Well, theoretically, if /lib was a symlink to /drive2/foo, then /lib/../lib would point to /drive2/lib if I'm not mistaken. Theoretically...
Edit: I just tested and it's not the case - it comes back to /lib. Hrm :(

Resources