According to g++ -print-search-dirs my C++ compiler is searching for libraries in many directories, including ...
/lib/../lib/:
/usr/lib/../lib/:
/lib/:
/usr/lib/
Naively, /lib/../lib/ would appear to be the same directory as /lib/ — lib's parent will have a child named lib, "that man's father's son is my father's son's son" and all that. The same holds for /usr/lib/../lib/ and /usr/lib/
Is there some reason, perhaps having to do with symbolic links, that g++ ought to be configured to search both /lib/../lib/ and /lib/?
If this is unnecessary redundancy, how would one go about fixing it?
If it matters, this was observed on an unmodified install of Ubuntu 9.04.
Edit: More information.
The results are from executing g++ -print-search-dirs with no other switches, from a bash shell.
Neither LIBRARY_PATH nor LPATH are output from printenv, and both echo $LPATH and echo LIBRARY_PATH return blank lines.
An attempt at an answer (which I gathered from a few minutes of looking at the gcc.c driver source and the Makefile environment).
These paths are constructed in runtime from:
GCC exec prefix (see GCC documentation on GCC_EXEC_PREFIX)
The $LIBRARY_PATH environment variable
The $LPATH environment variable (which is treated like $LIBRARY_PATH)
Any values passed to -B command-line switch
Standard executable prefixes (as specified during compilation time)
Tooldir prefix
The last one (tooldir prefix) is usually defined to be a relative path:
From gcc's Makefile.in
# Directory in which the compiler finds libraries etc.
libsubdir = $(libdir)/gcc/$(target_noncanonical)/$(version)
# Directory in which the compiler finds executables
libexecsubdir = $(libexecdir)/gcc/$(target_noncanonical)/$(version)
# Used to produce a relative $(gcc_tooldir) in gcc.o
unlibsubdir = ../../..
....
# These go as compilation flags, so they define the tooldir base prefix
# as ../../../../, and the one of the library search prefixes as ../../../
# These get PREFIX appended, and then machine for which gcc is built
# i.e i484-linux-gnu, to get something like:
# /usr/lib/gcc/i486-linux-gnu/4.2.3/../../../../i486-linux-gnu/lib/../lib/
DRIVER_DEFINES = \
-DSTANDARD_STARTFILE_PREFIX=\"$(unlibsubdir)/\" \
-DTOOLDIR_BASE_PREFIX=\"$(unlibsubdir)/../\" \
However, these are for compiler-version specific paths. Your examples are likely affected by the environment variables that I've listed above (LIBRARY_PATH, LPATH)
Well, theoretically, if /lib was a symlink to /drive2/foo, then /lib/../lib would point to /drive2/lib if I'm not mistaken. Theoretically...
Edit: I just tested and it's not the case - it comes back to /lib. Hrm :(
Related
When compiling code for coverage instrumentation (to use with lcov later on), we're compiling from a base directory tree (let's call it A), and we want the .gcda files to be produced at a different place (because the target directory tree is different - let's call it B).
So, the compilation command looked like this:
gcc -O0 -g --coverage -fprofile-dir=B -c -fPIC -Wall -o A/otherpath/to/mySourceFile.o A/path/to/mySourceFile.c
When checking the contents of mySourceFile.o (with the strings command), I saw that the mySourceFile.gcda file was set to be generated in B/A/otherpath/to/mySourceFile.gcda
Which is the mangling of the path given through the -fprofile-dir option with the exact absolute path of the object file created - just as written in the documentation. So far, no problem - except that what I want would be the mySourceFile.gcda file to be generated from the B directory, WITHOUT the A part.
So, the documentation also mentions the -fprofile-prefix-path option, which is supposed to allow you to remove part of the path, so that the mangling doesn't add the old path to the new.
I tried using it in the following way:
gcc -O0 -g --coverage -fprofile-dir=B -fprofile-prefix-path=A -c -fPIC -Wall -o A/otherpath/to/mySourceFile.o A/path/to/mySourceFile.c
However, after checking through strings, once again, in the generated object file, the path was still B/A/otherpath/to/mySourceFile.gcda, whereas I expected it to be B/otherpath/to/mySourceFile.gcda (that is, I expected the A part to have been stripped by the -fprofile-prefix-path option.)
Obviously, it didn't work. Any insight why ?
( Compiler used is GCC 11.2.1, which is a version recent enough to know about the option. )
Ok, after some tinkering, I got results. Maybe not exactly what I was expecting, but close enough.
Let me start by saying that the A and B "directories" I mentioned in my question were absolute paths. And it didn't work well.
However, while keeping the absolute B (target) path, I tried not using the full A (source) path while compiling. More precisely, I didn't use it to specify the OUTPUT file name, for the object. Instead, I went to the base directory (the A path), and then, ran the command by specifying the output file path relative to the current (A) directory
Which would give the following command:
(From directory A)
gcc -O0 -g --coverage -fprofile-dir=B -fprofile-prefix-path=A -c -fPIC -Wall -o otherpath/to/mySourceFile.o path/to/mySourceFile.c
This time, the source command did show an interesting result, for the mySourceFile.gcda file:
B#otherpath#to#mySourceFile.gcda
As you can see, it's not exactly what I wanted (there are # instead of /), but mentions to A disappeared, and overall, I'm confident it should work as intended. Not utterly sure yet (I still have to test it on the target platform, which will need tinkering with the way the makefiles currently work), but confident nonetheless.
Also, if I didn't use the -fprofile-prefix-path in the command, then the string would mention the A path, like this (with the '/' inside the A path being replaced with '#' characters, obviously):
B#A#otherpath#to#mySourceFile.gcda
So, the option works, but only when using relative paths, not when using absolute ones, for the object file. Hope that helps people.
PS: I checked by changing the path to the source (.c) file. Whether specified using absolute, or relative, paths, it didn't change the outcome. What matters is specifying the path to the object file in a relative manner.
I wrote a library libfoo providing functions bar and baz.
I want the user to be able to find the same man-page (from mans/libfoo.man) when they call man libfoo, man bar and man baz (Similar to man fprintf, man sprintf all pointing to the same page.)
My current setup has the files mans/libfoo.man and Makefile.am
To 'tell' automake that I want to end up with the three man-pages I specified the dist_man3_MANS variable.
Makefile.am:
dist_man3_MANS = mans/libfoo.3 mans/bar.3 mans/baz.3
Coming from GNU make, I thought I could just write
%.3: mans/libfoo.man
ln -S libfoo.man $#
to create links temporarily and then let Automake install those accordingly, but Automake errors out with Makefile.am:115: warning: '%'-style pattern rules are a GNU make extension. I want to do it properly and take this warning seriously by not relying on GNU Make to be as portable as possible.
The Automake manual suggests to add a target
.man.3:
$(LN_S) $^ $#
but that just tells Automake that xx.man can be compiled to xx.3, requiring the base name to be the same. I don't want to carry around those xx.man files, so this approach does not work.
I could hack it in with putting a rule
dist_man3_MANS = mans/libfoo.3 mans/bar.3 mans/baz.3
$(dist_man3_MANS): mans/libfoo.man
$(LN_S) libfoo.man $#
but that seems like a dirty hack, because I am not giving it a recipe to compile .man to .3, but rather say: "Hey, you can create those files with this rule", which for this case may work coincidental.
I would follow the example from the Automake info page section Extending Automake Rules and do something along the lines of
LIBFOO_MAN_ALIASES = bar baz
install-data-hook:
set -e; \
cd $(DESTDIR)$(man3dir) && \
for manalias in $(LIBFOO_MAN_ALIASES); do \
$(LN_S) libfoo.3 $${manalias}.3; \
done
uninstall-hook:
cd $(DESTDIR)$(man3dir) && \
for manalias in $(LIBFOO_MAN_ALIASES); do \
rm -f $${manalias}.3; \
done
relying on AC_PROG_LN_S to make sure that $(LN_S) does something reasonable for the system (symlink, hardlink, copy) to create a file name which can be open(2)ed and read.
FTR, I have just taken a look at three different systems' man pages and found them using three different methods to make the fprintf(3) man page show the same man page as printf(3) does:
Debian 10 uses symlinks
Fedora 35 uses a /usr/share/man/man3/fprintf.3 file containing .so man3/printf.3 (while some other man pages use symlinks to achieve the same effect)
FreeBSD 13 uses hardlinks, and find /usr/share/man -type l does not find any symlinks on my relatively clean system. However, manually testing both symlinks and the .so man3/printf.3 method suggests that FreeBSD man(1) does not treat symlinks in any special way and therefore opens the symlinked man page, and it also interprets the .so command just like Fedora 35's man(1) does.
I do not know how portable each of those methods is. Each of these three methods could set up on make install by using an appropriate install-data-hook, but any man file which can be opened using open(2) appears to be work, and therefore $(LN_S) looks like a good bet.
Like
make,
scons has a large number of predefined variables and rules. (Try scons | wc on an SConstruct containing env = Environment(); print(env.Dump()) to see how extended the set is.)
But suppose we aren't after the wizardry of presets but rather want to do something a lot more primitive—simulating launching a few instructions from the (bash, etc) command line?
Also suppose we're quite happy with the default Decider('MD5'). What is the translation of the one-souce-one-target:
out/turquoise.xyz: out/chartreuse.xyz
chartreuse_to_turquoise $< $#
of the two-source-one-target:
out/purple.xyz: out/lilac.xyz out/salmon.xyz
gen_purple $< $#
and of:
run_this:
python prog.py
which we would run on-demand by typing make run_this?
What does the SConstruct for these elementary constructs look like?
All the answers you're looking for are in the users guide (and manpage)
Firstly, assuming you don't want to scan the input files to add included files specified in the input files, you can use Commmand()
(See info here: https://scons.org/doc/production/HTML/scons-user.html#chap-builders-commands)
Then you'll want an alias to specify an a non file command line target
(See here:https://scons.org/doc/production/HTML/scons-user.html#chap-alias)
Putting those two together yields
env=Environment()
# one source, one target
env.Command('out/turquoise.xyz', 'out/chartreuse.xyz', 'chartreuse_to_turquoise $SOURCE $TARGET')
# Two source, one target
env.Command('out/purple.xyz',['out/lilac.xyz','out/salmon.xyz'], 'gen_purple $SOURCES $TARGET')
# And your .phony make target which is actually not great for reproducibility and determining when it should be rerun, because you do not specify any sources or targets
env.Alias('run_this','python prog.py')
Note: SCons doesn't NOT propagate your shell environment variables. So if you depend on (for example) a non system path in your PATH, you'll need to explicitly specify that in env['ENV']['PATH'] for example. For more details take a read through the users guide, manpage and FAQ.
https://scons.org/doc/production/HTML/scons-user.html
https://scons.org/doc/production/HTML/scons-man.html
https://scons.org/faq.html
And you can reach the community directly via our discord server, IRC channel, or users mailing list
When I exec qemu-aarch64 with a binary which is using shared libraries I get the following:
qemu-aarch64 -L /usr/aarch64-linux-gnu ./test
./test: error while loading shared libraries: testlibrary.so.1: cannot open shared object file: No such file or directory
Obviously it is because test does not know where the shared libraries are.
Thus:
qemu-aarch64 -L /usr/aarch64-linux-gnu -E LD_PRELOAD="/home/test/libraries/testlibrary.so.1 /home/test/libraries/testlibrary2.so.1" ./test
hi!
Ok, it works; but when I use LD_LIBRARY_PATH it does not work:
qemu-aarch64 -L /usr/aarch64-linux-gnu -E LD_LIBRARY_PATH="/home/test/libraries ./test
./test: error while loading shared libraries: testlibrary.so.1: cannot open shared object file: No such file or directory
The difference between LD_PRELOAD and LD_LIBRARY_PATH, from ld.so man:
LD_PRELOAD:
A list of additional, user-specified, ELF shared objects
to be loaded before all others. This feature can be used
to selectively override functions in other shared objects.
The items of the list can be separated by spaces or
colons, and there is no support for escaping either
separator. The objects are searched for using the rules
given under DESCRIPTION. Objects are searched for and
added to the link map in the left-to-right order specified
in the list.
In secure-execution mode, preload pathnames containing
slashes are ignored. Furthermore, shared objects are
preloaded only from the standard search directories and
only if they have set-user-ID mode bit enabled (which is
not typical).
Within the names specified in the LD_PRELOAD list, the
dynamic linker understands the tokens $ORIGIN, $LIB, and
$PLATFORM (or the versions using curly braces around the
names) as described above in Dynamic string tokens. (See
also the discussion of quoting under the description of
LD_LIBRARY_PATH.)
There are various methods of specifying libraries to be
preloaded, and these are handled in the following order:
(1) The LD_PRELOAD environment variable.
(2) The --preload command-line option when invoking the
dynamic linker directly.
(3) The /etc/ld.so.preload file (described below).
And,
LD_LIBRARY_PATH:
A list of directories in which to search for ELF libraries
at execution time. The items in the list are separated by
either colons or semicolons, and there is no support for
escaping either separator. A zero-length directory name
indicates the current working directory.
This variable is ignored in secure-execution mode.
Within the pathnames specified in LD_LIBRARY_PATH, the
dynamic linker expands the tokens $ORIGIN, $LIB, and
$PLATFORM (or the versions using curly braces around the
names) as described above in Dynamic string tokens. Thus,
for example, the following would cause a library to be
searched for in either the lib or lib64 subdirectory below
the directory containing the program to be executed:
$ LD_LIBRARY_PATH='$ORIGIN/$LIB' prog
(Note the use of single quotes, which prevent expansion of
$ORIGIN and $LIB as shell variables!)
Why does it work with LD_PRELOAD and not with LD_LIBRARY_PATH?
The library you're opening with LD_PRELOAD is "testlibrary.so.1", but the library that the dynamic loader otherwise looks for is "testlibrary.so.3", which suggests there's a mismatch between the library version you have and the library version the binary is linked against, which maybe the LD_PRELOAD is side-stepping. Does LD_LIBRARY_PATH work if you make sure that you have the file in that directory that the binary is looking for ?
I have a Linux Qt program. I'd like it to preferentially use the (dynamic) Qt libraries in the executable's directory if they exist, otherwise use the system's Qt libs. RPATH to the rescue.
I add this line to the qmake's .pro file:
QMAKE_LFLAGS += '-Wl,-rpath,\'\$$ORIGIN\''
and looking at the resulting executable with readelf I see:
0x000000000000000f (RPATH) Library rpath: [$ORIGIN:/usr/local/Trolltech/Qt-5.2.0/lib]
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN:/usr/local/Trolltech/Qt-5.2.0/lib]
Seems right, but ldd shows it's using the system version:
libQt5Core.so.5 => /usr/local/Trolltech/Qt-5.2.0/lib/libQt5Core.so.5 (0x00007f2d2fe09000)
If I manually edit qmake's resulting Makefile to swap the order of the two rpaths, so $ORIGIN comes after /usr/local/..., I get the right behavior:
0x000000000000000f (RPATH) Library rpath: [/usr/local/Trolltech/Qt-5.2.0/lib:$ORIGIN]
0x000000000000001d (RUNPATH) Library runpath: [/usr/local/Trolltech/Qt-5.2.0/lib:$ORIGIN]
libQt5Core.so.5 => ./libQt5Core.so.5 (0x00007fb92aba9000)
My problem is with how qmake constructs the final LFLAGS variable. I can't figure out how to make it put my addition ($ORIGIN) after the system library. Any ideas?
You can add the following to your .pro file to force the dynamic linker to look in the same directory as your Qt application at runtime in Linux :
unix:{
# suppress the default RPATH if you wish
QMAKE_LFLAGS_RPATH=
# add your own with quoting gyrations to make sure $ORIGIN gets to the command line unexpanded
QMAKE_LFLAGS += "-Wl,-rpath,\'\$$ORIGIN\'"
}
If you want it to look in a subdirectory of the executable path, you can use :
QMAKE_LFLAGS += "-Wl,-rpath,\'\$$ORIGIN/libs\'"
Note that you should have the .so files with the exact same name in your application directory. For example you should copy libQt5Core.so.5.2.0 to your application directory with the name libQt5Core.so.5. Now the ldd shows the directory of the application.
You can also have libQt5Core.so.5.2.0 and a link to it with the name libQt5Core.so.5 in the application directory.
As far as my research can say, you can only add RPATH at the beginning of the list with QMake.
But if you are on Linux and can install chrpath, you can hack your way around that.
Add this block at the end of your .pro file
# Add spacing since chrpath cannot expand RPATH length
QMAKE_RPATHDIR = \
/XYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXY1\
/XYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXY2\
/XYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXY3\
/XYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXYXY4
QMAKE_POST_LINK += 'chrpath -r \'/my/qt/installation:\$$ORIGIN\' $$OUT_PWD/mybinaryname;'
I'm taking a bit of a guess at what's happening, but it's based on knowing some of the odd behaviours of ld.
check for the presence of an LD_LIBRARY_PATH variable that will come into effect before the processing of a RUNPATH variable. Because of the presence of both RPATH and RUNPATH, the LD_LIBRARY_PATH rule comes into effect, so if it's set then unset it.
Secondly, I'd never expect to see:
libQt5Core.so.5 => ./libQt5Core.so.5 (0x00007fb92aba9000)
in the output of ldd, I would always see the expansion of $ORIGIN to the directory of the binary (maybe you shortened it?), so I would have expected:
libQt5Core.so.5 => /path/to/bin/./libQt5Core.so.5 (0x00007fb92aba9000)
Which means it sounds like the LD_LIBRARY_PATH expansion is .:/usr/local/Trolltech/Qt-5.2.0/lib, which to me sounds like you've got environmental overrides happening.
qmake would always append the QMAKE_RPATHDIR with the QT_INSTALL_LIBS internally defined in $(QT_DIR)/mkspecs/features/qt.prf file:
170: relative_qt_rpath:!isEmpty(QMAKE_REL_RPATH_BASE):contains(INSTALLS, target):\
173: QMAKE_RPATHDIR += $$relative_path($$[QT_INSTALL_LIBS], $$qtRelativeRPathBase())
175: QMAKE_RPATHDIR += $$[QT_INSTALL_LIBS/dev]
179:!isEmpty(QMAKE_LFLAGS_RPATHLINK):!contains(QT_CONFIG, static) {
189: QMAKE_RPATHLINKDIR *= $$unique(rpaths)
So to avoid your application using the QT library from system path, comment out the lines above which append the QMAKE_RPATHDIR and add QMAKE_RPATHDIR=$ORIGIN into your .pro file.