How can get g++ to use my own glibc build's headers correctly? - linux

Question

There's a TL;DR at the end if the context is too much!
Context
I am trying to update the version of glibc a project uses to 2.23 (I know it's old, that's another issue). In order to do this, I need to swap out the libraries and use the associated interpreter.
I encountered some issues when swapping out the interpreter that looked like an ABI change, so I figured it was probably because the header files had changed somehow and started working on getting those included into the project.
At first I tried using -I to include the headers, but got an error (see below). Later I tried setting --sysroot, but this quickly felt like the wrong way of doing things since I was essentially reinventing what g++ already did with system headers. I later found another mechanism that looked more promising (see Problem section).
Could this be an XY issue? Absolutely, but either way, the problem I'm seeing seems odd to me.
Problem
I looked into whether there was a different mechanism to include headers for system libraries, such as glibc, in gcc and g++. I found the flag -isystem:
-isystem dir
Search dir for header files, after all directories specified by -I but before the standard system directories. Mark it as a system directory, so that it gets the same special treatment as is applied to the standard system directories. If dir begins with "=", then the "="
will be replaced by the sysroot prefix; see --sysroot and -isysroot.
I figured that this was probably wanted and set about intergrating this flag into the build system for the project. The resulting g++ command looks like this (simplified and broken onto multiple lines):
> /path/to/gcc-6.3.0/bin/g++
-c
-Wl,--dynamic-linker=/path/to/glibc-2.23/build/install/lib/ld-linux-x86-64.so.2
-Wl,--rpath=/path/to/glibc-2.23/build/install/lib
-isystem /path/to/glibc-2.23/build/install/include
-I.
-I/project-foo/include
-I/project-bar/include
-o example.o
example.cpp
This leads to the following error, followed by many similar ones:
In file included from /usr/include/math.h:71:0,
from /path/to/gcc-6.3.0/include/c++/6.3.0/cmath:45,
from example.cpp:42:
/path/to/glibc-2.23/build/install/include/bits/mathcalls.h:63:16: error: expected constructor, destructor, or type conversion before '(' token
__MATHCALL_VEC (cos,, (_Mdouble_ __x));
Looking into this, it appears that this particular math.h is incompatible with this version of glibc. The fact it tries to use it surprises me, because the math.h file exists in the glibc directory I specified; why didn't it use that? Here's how I verified that file exists:
> ls /path/to/glibc-2.23/build/install/include/math.h
/path/to/glibc-2.23/build/install/include/math.h
Research
I searched around on the internet for people with a similar issue and came across the following relevant things:
https://github.com/riscv/riscv-gnu-toolchain/issues/105
https://askubuntu.com/questions/806220/building-ucb-logo-6-errors-in-mathcalls-h
-isystem on a system include directory causes errors
The last of these is the most promising; it talks about why -isystem won't work here stating that the special #include_next traverses the include path in a different way. Here, the solution appears to be "don't use -isystem where you can help it", but since I've tried using -I only get get the same problem again, I'm not sure how I'd apply that here.
Original issue
When compiling with the new glibc, I get the following error (our build process ends up running some of the programs it compiles to generate further source to be compiled, hence this runtime error whilst compiling):
Inconsistency detected by ld.so: get-dynamic-info.h: 143: elf_get_dynamic_info: Assertion `info[DT_RPATH] == NULL' failed!
I found a couple of relevant things about this:
https://www.linuxquestions.org/questions/linux-software-2/how-to-get-local-gcc-to-link-with-local-glibc-404087/
https://www.linuxquestions.org/questions/programming-9/inconsistency-detected-by-ld-so-dynamic-link-h-62-elf_get_dynamic_info-assertion-621701/
The only solution I see there is completely recompiling gcc to use the new glibc. I'd like to avoid that if possible, which is what lead me down the include route.
Eliminating the complex build system
To try and eliminate the complex build system on the "real" project, I reproduced the problem using the following test.cpp file:
#include <cmath>
int main() {
}
Compiled using:
> /path/to/gcc-6.3.0/bin/g++ test.cpp -Wl,--dynamic-linker=/path/to/glibc-2.23/build/install/lib/ld-linux-x86-64.so.2 -Wl,--rpath=/path/to/glibc-2.23/build/install/lib
Running yields the same original issue:
> ./a.out
Inconsistency detected by ld.so: get-dynamic-info.h: 143: elf_get_dynamic_info: Assertion `info[DT_RPATH] == NULL' failed!
Trying to use the newer headers yields the same include issue:
> /path/to/gcc-6.3.0/bin/g++ test.cpp -Wl,--dynamic-linker=/path/to/glibc-2.23/build/install/lib/ld-linux-x86-64.so.2 -Wl,--rpath=/path/to/glibc-2.23/build/install/lib -isystem /path/to/glibc-2.23/build/install/include
In file included from /usr/include/math.h:71:0,
from /path/to/gcc-6.3.0/include/c++/6.3.0/cmath:45,
from test.cpp:1:
/path/to/glibc-2.23/build/install/include/bits/mathcalls.h:63:16: error: expected constructor, destructor, or type conversion before '(' token
__MATHCALL_VEC (cos,, (_Mdouble_ __x));
TL;DR
How can I get g++ to include the headers from my glibc build correctly, without it accidentally including incompatible files from /usr/include?

Answer 1

In your GCC version,<cmath> uses #include_next, which means that you need to make sure that the directory which contains the cmath file comes before (on the include search path) the directory with the proper math.h for the version of glibc you are building against.
You can use g++ -v to view the search path. In your case, it probably looks like this:
#include "..." search starts here:
#include <...> search starts here:
.
/project-foo/include
/project-bar/include
/path/to/glibc-2.23/build/install/include
/usr/include/c++/6
/usr/include/x86_64-linux-gnu/c++/6
/usr/lib/gcc/x86_64-linux-gnu/6/include
/usr/local/include
/usr/lib/gcc/x86_64-linux-gnu/6/include-fixed
/usr/include/x86_64-linux-gnu
/usr/include
If you configure glibc with --prefix=/usr and install it with DESTDIR=/path/to/glibc-2.23/build/install, its header files will be installed into the directory /path/to/glibc-2.23/build/install/usr/include. This means you should be able to use the -isysroot option, which rewrites the default /usr/include directory, resulting in the right ordering of the search path:
#include "..." search starts here:
#include <...> search starts here:
.
/project-foo/include
/project-bar/include
/usr/include/c++/6
/usr/include/x86_64-linux-gnu/c++/6
/usr/include/c++/6/backward
/usr/lib/gcc/x86_64-linux-gnu/6/include
/usr/lib/gcc/x86_64-linux-gnu/6/include-fixed
/path/to/glibc-2.23/build/install/usr/include