Get rid of "gcc - /usr/bin/ld: warning lib not found" - linux

I have the following warning during link:
/usr/bin/ld: warning: libxxx.so.6, needed by /a/b/c/libyyy.so, not found (try using -rpath or -rpath-link)
Setting environment variable LD_LIBRARY_PATH=path_to_libxxx.so.6 silence the warning (adding -Lpath_to_libxxx.so.6 doesn't help).
I have a separate compilation server, where the resulting binary is only compile.
The binary is executed on other server and there the libxxx.so.6 is seen by the binary (checked with ldd executable).
Is there're other way to get rid of the warning at compilation time (I have it several times and it's very annoying)?

You need to add the dynamic library equivalent of -L:
-Wl,-rpath-link,/path/to/lib
This will cause the linker to look for shared libraries in non-standard places, but only for the purpose of verifying the link is correct.
If you want the program to find the library at that location at run-time, then there's a similar option to do that:
-Wl,-rpath,/path/to/lib
But, if your program runs fine without this then you don't need it.

Make sure the paths to the needed libraries are known to the runtime linker. This is done by adding a file in /etc/ld.so.conf.d/ with the needed path. For example, /etc/ld.so.conf.d/foo with the following contents:
/usr/local/lib/foo/
If you have a very old Linux version, /etc/ld.so.conf.d/ might not be supported, in which case you might have to add the paths directly into the /etc/ld.so.conf file.
After you've done that, you need to update the linker's database by executing the "ldconfig" command.

I know this is old, but here's a better fix:
The root cause:
The problem actually happens when LD invoked by GCC starts resolving
library dependencies. Both GCC and LD are aware of the sysroot
containing libraries, however LD may be missing one critical
component: the /etc/ld.so.conf file. Here’s an exampleld.so.conf file
from a Raspberry PI system:
include /etc/ld.so.conf.d/*.conf
The /etc/ld.so.conf.d directory contains the following files:
00-vmcs.conf:
/opt/vc/lib
arm-linux-gnueabihf.conf:
/lib/arm-linux-gnueabihf /usr/lib/arm-linux-gnueabihf
libc.conf:
/usr/local/lib
The universal solution
 
The problem can be easily solved by copying the LD configuration files
to a location where the cross-toolchain’s LD can find them. There’s
one pitfall however: if your cross-toolchain was built with MinGW
(most are), it probably did not have access to the glob() function so
it won’t be able to parse a wildcard-enabled include statement
like *.conf. The workaround here is to just manually combine the
contents of all .conf files from /etc/ld.so.conf.d and paste them
into /etc/ld.so.conf
*/opt/vc/lib
/lib/arm-linux-gnueabihf
/usr/lib/arm-linux-gnueabihf
/usr/local/lib*
Once you create the ld.so.conf file in the correct folder, your
toolchain will be able to resolve all shared library references
automatically and you won’t see that error message again!

The only way to silence these warning using command line options would be the -L flag which curiously does not work for you (maybe you can post more details on this). Since the warning is generated by ld we could try to use -Wl,option to disable a linker warning but from the documentation of GNU ld however there is no option for (de)activating this warnings.
So this leaves us with writing a wrapper script filtering out this warning or compile a custom version of ld.

Related

Is it possible to permanently add directories to the default ld search path?

I'm attempting to install a library that will be used generally on this node. Currently the library is installed in a location that includes it's version name /opt/sample-x.y.z/lib/libsample.so. Ideally I'd like to be able to update the library, change the sys configs and not bother other devs with needing to change paths on trivial changes. I've added the library directory to the /etc/ld.so.conf.d/sample.conf and run ldconfig, but this only seems to affect loading not linking.
When I run ldconfig -v | grep sample, I correctly see libsample.so.
However, whenever I run ld -lsample --verbose, it fails to find the library. In the verbose output, it details the directories that it searches and the one I added with ldconfig isn't there. As a workaround, I tested adding a symlink to the library in /usr/lib and ld was able to find it.
Are there other ways to add the library to the default linker path? Ideally I'm looking for a config file that I can modify similar to the one for loading.
If it matters, the node is a Centos7 node.
No, it is not possible. The paths that ld uses during the linking phase are configured via scripts which are pulled into ld during its build process. There isn't a way to update these after ld is built. There are other questions and answers related to accomplishing this for GCC specifically which seem to involve the LIBRARY_PATH environment variable. That solution, however, is gcc specific.

How can get g++ to use my own glibc build's headers correctly?

There's a TL;DR at the end if the context is too much!
Context
I am trying to update the version of glibc a project uses to 2.23 (I know it's old, that's another issue). In order to do this, I need to swap out the libraries and use the associated interpreter.
I encountered some issues when swapping out the interpreter that looked like an ABI change, so I figured it was probably because the header files had changed somehow and started working on getting those included into the project.
At first I tried using -I to include the headers, but got an error (see below). Later I tried setting --sysroot, but this quickly felt like the wrong way of doing things since I was essentially reinventing what g++ already did with system headers. I later found another mechanism that looked more promising (see Problem section).
Could this be an XY issue? Absolutely, but either way, the problem I'm seeing seems odd to me.
Problem
I looked into whether there was a different mechanism to include headers for system libraries, such as glibc, in gcc and g++. I found the flag -isystem:
-isystem dir
Search dir for header files, after all directories specified by -I but before the standard system directories. Mark it as a system directory, so that it gets the same special treatment as is applied to the standard system directories. If dir begins with "=", then the "="
will be replaced by the sysroot prefix; see --sysroot and -isysroot.
I figured that this was probably wanted and set about intergrating this flag into the build system for the project. The resulting g++ command looks like this (simplified and broken onto multiple lines):
> /path/to/gcc-6.3.0/bin/g++
-c
-Wl,--dynamic-linker=/path/to/glibc-2.23/build/install/lib/ld-linux-x86-64.so.2
-Wl,--rpath=/path/to/glibc-2.23/build/install/lib
-isystem /path/to/glibc-2.23/build/install/include
-I.
-I/project-foo/include
-I/project-bar/include
-o example.o
example.cpp
This leads to the following error, followed by many similar ones:
In file included from /usr/include/math.h:71:0,
from /path/to/gcc-6.3.0/include/c++/6.3.0/cmath:45,
from example.cpp:42:
/path/to/glibc-2.23/build/install/include/bits/mathcalls.h:63:16: error: expected constructor, destructor, or type conversion before '(' token
__MATHCALL_VEC (cos,, (_Mdouble_ __x));
Looking into this, it appears that this particular math.h is incompatible with this version of glibc. The fact it tries to use it surprises me, because the math.h file exists in the glibc directory I specified; why didn't it use that? Here's how I verified that file exists:
> ls /path/to/glibc-2.23/build/install/include/math.h
/path/to/glibc-2.23/build/install/include/math.h
Research
I searched around on the internet for people with a similar issue and came across the following relevant things:
https://github.com/riscv/riscv-gnu-toolchain/issues/105
https://askubuntu.com/questions/806220/building-ucb-logo-6-errors-in-mathcalls-h
-isystem on a system include directory causes errors
The last of these is the most promising; it talks about why -isystem won't work here stating that the special #include_next traverses the include path in a different way. Here, the solution appears to be "don't use -isystem where you can help it", but since I've tried using -I only get get the same problem again, I'm not sure how I'd apply that here.
Original issue
When compiling with the new glibc, I get the following error (our build process ends up running some of the programs it compiles to generate further source to be compiled, hence this runtime error whilst compiling):
Inconsistency detected by ld.so: get-dynamic-info.h: 143: elf_get_dynamic_info: Assertion `info[DT_RPATH] == NULL' failed!
I found a couple of relevant things about this:
https://www.linuxquestions.org/questions/linux-software-2/how-to-get-local-gcc-to-link-with-local-glibc-404087/
https://www.linuxquestions.org/questions/programming-9/inconsistency-detected-by-ld-so-dynamic-link-h-62-elf_get_dynamic_info-assertion-621701/
The only solution I see there is completely recompiling gcc to use the new glibc. I'd like to avoid that if possible, which is what lead me down the include route.
Eliminating the complex build system
To try and eliminate the complex build system on the "real" project, I reproduced the problem using the following test.cpp file:
#include <cmath>
int main() {
}
Compiled using:
> /path/to/gcc-6.3.0/bin/g++ test.cpp -Wl,--dynamic-linker=/path/to/glibc-2.23/build/install/lib/ld-linux-x86-64.so.2 -Wl,--rpath=/path/to/glibc-2.23/build/install/lib
Running yields the same original issue:
> ./a.out
Inconsistency detected by ld.so: get-dynamic-info.h: 143: elf_get_dynamic_info: Assertion `info[DT_RPATH] == NULL' failed!
Trying to use the newer headers yields the same include issue:
> /path/to/gcc-6.3.0/bin/g++ test.cpp -Wl,--dynamic-linker=/path/to/glibc-2.23/build/install/lib/ld-linux-x86-64.so.2 -Wl,--rpath=/path/to/glibc-2.23/build/install/lib -isystem /path/to/glibc-2.23/build/install/include
In file included from /usr/include/math.h:71:0,
from /path/to/gcc-6.3.0/include/c++/6.3.0/cmath:45,
from test.cpp:1:
/path/to/glibc-2.23/build/install/include/bits/mathcalls.h:63:16: error: expected constructor, destructor, or type conversion before '(' token
__MATHCALL_VEC (cos,, (_Mdouble_ __x));
TL;DR
How can I get g++ to include the headers from my glibc build correctly, without it accidentally including incompatible files from /usr/include?
In your GCC version,<cmath> uses #include_next, which means that you need to make sure that the directory which contains the cmath file comes before (on the include search path) the directory with the proper math.h for the version of glibc you are building against.
You can use g++ -v to view the search path. In your case, it probably looks like this:
#include "..." search starts here:
#include <...> search starts here:
.
/project-foo/include
/project-bar/include
/path/to/glibc-2.23/build/install/include
/usr/include/c++/6
/usr/include/x86_64-linux-gnu/c++/6
/usr/lib/gcc/x86_64-linux-gnu/6/include
/usr/local/include
/usr/lib/gcc/x86_64-linux-gnu/6/include-fixed
/usr/include/x86_64-linux-gnu
/usr/include
If you configure glibc with --prefix=/usr and install it with DESTDIR=/path/to/glibc-2.23/build/install, its header files will be installed into the directory /path/to/glibc-2.23/build/install/usr/include. This means you should be able to use the -isysroot option, which rewrites the default /usr/include directory, resulting in the right ordering of the search path:
#include "..." search starts here:
#include <...> search starts here:
.
/project-foo/include
/project-bar/include
/usr/include/c++/6
/usr/include/x86_64-linux-gnu/c++/6
/usr/include/c++/6/backward
/usr/lib/gcc/x86_64-linux-gnu/6/include
/usr/lib/gcc/x86_64-linux-gnu/6/include-fixed
/path/to/glibc-2.23/build/install/usr/include

Bash command: export BLAS_LIBS="-L$LAPACKHOME/lib -lblas"

Can any body explain to me what does the whole sentence mean?
I know this is to set Macro BLAS_LIBS as another string.
But I'm not sure what's the "-lblas" mean and I don't know how to use it.
Similar as the following code. "-llapack"
export LAPACK_LIBS="-L$LAPACKHOME/lib -llapack"
How can the program find out the BLAS and LAPACK libraries just by "-lblas" and "-llapack" ?
Thanks for advance.
I'm not sure why you say "just by -llapack" because that's not what is happening here. Specifically, the -L option just before it specifies a directory path to add to the library resolution path. This works roughly like PATH in the shell.
For example, with the command line fragment gcc -Lfoodir -Lbardir -lfoo -lbar, you basically instruct the linker to search the directories foodir and bardir for the library files libfoo.a and libbar.a.
The -l option is described in GCC: Options for Linking and -L and friends in the following section GCC: Options for Directory Search.
This build arrangement -- configure the build to show where the required files are before compiling -- is common for libraries, where if a user has already downloaded and compiled a required library for some other project, they don't need to rebuild it; they can just point the compiler to wherever they already have the stuff needed for this project.
Building your own libraries is becoming increasingly unnecessary anyway, as prepackaged binaries of most common libraries are available for most systems these days. But of course, if you are on an unusual platform, or have specialized needs which dictate recompilation with different options than any available prebuilt binary, you will still need to understand how to do this.

ld linker script error

The project I'm currently working on uses a bunch of dynamic libs bundled together with the source code. There is a subset of libs which is named as follows:
for a given lib libABC, there are four files:
libABC.so
libABC.so.4
libABC.so.4.5
libABC.so.4.5.0
They are daisy-chained like this:
the first file, libABC.so, contains the following:
link libABC.so.4
whereas the next file, libABC.so.4, contains the following:
link libABC.so.4.5
and so on till the actual lib file, libABC.so.4.5.0.
I know that this kind of stuff should be done using symlinks, but we can't change that, it's a commercial project. So the linker chokes on that!
/usr/bin/ld: path/to/the/packaged/libs/libABC.so:unrecognized file format, treating as linker script
(which it actually is, heh)
/usr/bin/ld: path/to/the/packaged/libs/libABC.so:1:syntax error
Now I can't seem to find any info on the GNU ld linker script command "link" or any complete reference to the GNU ld linker script commands.
What could that be?
The linker script format is described in the GNU ld manual.
I found this issue myself today, and what happens is when you do a checkout using SVN on Windows (e.g. TortoiseSVN), the Linux symlink is converted into a text file.
Then, on the build process, ld is receiving a text file instead of a symlink that would lead to the concrete file.
Solution: svn checkout on the Linux machine.

shared library problems on linux

I'm trying to compile/link a very old piece of software on a linux system and I can't for some reason link with a shared library that's installed on my system.
I get the following error from the linker:
/usr/bin/ld: cannot find -lXaw
However, the lib itself is installed. If I run
ldconfig -v | grep libXaw
I get (among other things) this hit:
libXaw.so.7 -> libXaw7.so.7.0.0
The library and the links to it are in /usr/lib btw. So nothing special.
So the library is there and ldconfig finds it. What could ld cause ld from not finding the library during link-time? As you may have already guessed I'm quite new to the shared library stuff.
Any ideas?
The linker may be looking, literally, for "libXaw.so". Is that in /usr/lib? If not, you could try adding it as another soft link from libXaw7.so.7.0.0.
The reason for the symlink btw is to select the default version to link against in the case of multiple versions, keep in mind the name of the library is integrated into the binary. (which you can see with ldd).
Are the -L library directories being overridden, and it's not looking in /usr/lib?
To link it, you need the .a file, NOT the .so file, which is the runtime library. The shared object is only useful to a program already linked against the non-shared parts of a library. This is typically distributed in a ".a" file.

Resources