I have run into a problem when compiling with -flto in that I run into a problem where a symbol appears to disappear in a shared library which causes an undefined symbol error when attempting to dlopen another shared library. This only happens when -flto is defined and happens with g++ 7.5 and 11.2.
When I look at the shared library which contains the C++ template that's failing with lto with readelf -Ws, I see some curious differences.
Without LTO, I see these two entries, indicating a weak function:
3805: 0000000000334a70 1843 FUNC WEAK DEFAULT 13 _ZN13VirtualMapper13stage2_lookupILb0EEEN3vms15lookup_result_tEimbN6sysreg9ESR_ELX_tENS_8Access_tEb
2660: 0000000000334a70 1843 FUNC WEAK DEFAULT 13 _ZN13VirtualMapper13stage2_lookupILb0EEEN3vms15lookup_result_tEimbN6sysreg9ESR_ELX_tENS_8Access_tEb
I am not sure why the function is being marked as weak since the attribute isn't being specified in the code.
However, with -flto enabled, I instead see the following:
728: 000000000007e4d2 1422 FUNC LOCAL DEFAULT 13
_ZN13VirtualMapper13stage2_lookupILb0EEEN3vms15lookup_result_tEimbN6sysreg9ESR_ELX_tENS_8Access_tEb.cold
1308: 0000000000303430 7449 FUNC LOCAL DEFAULT 13 _ZN13VirtualMapper13stage2_lookupILb0EEEN3vms15lookup_result_tEimbN6sysreg9ESR_ELX_tENS_8Access_tEb
In this case, instead of being marked as weak, it has ".cold" appended to it.
Furthermore, when I attempt to dump using objdump -T -w, in the working case without lto I see:
0000000000334a70 w DF .text 0000000000000733 Base _ZN13VirtualMapper13stage2_lookupILb0EEEN3vms15lookup_result_tEimbN6sysreg9ESR_ELX_tENS_8Access_tEb
However, with lto enabled, this symbol is not present when listed with objdump.
I cannot provide source since it is from a very large proprietary c++ project, unfortunately. It is compiled using -fPIC -shared -rdynamic -mcx16 -march=native (on AMD Threadripper) along with the proper rpath assigned.
As for the CPPFLAGS, -O3, -march=native -fPIC -mno-red-zone -std=gnu++11 and various parameters to enable various warnings. I had to compile with -fPIC instead of -fpic due to the compiler complaining. Note that this is also with clean builds as well.
This is with OpenSUSE 15.3 with libc 2.31 and libstdc++ 6.0.29. This happens with both g++ 7.5 and 11.2. Everything works fine when lto is not enabled.
Any idea on what could be going on? To me this looks like a bug in the lto optimizer. Symbols should not disappear in shared libraries when lto is enabled.
The problem turned out that the template needed to be explicitely instantiated in the .cpp file where it was defined. Without that, the compiler appears to have attempted to inline it and not make it externally visible. The other option is to add attribute((externally_visible)) to the template definition in the header file.
Related
Does the gcc's option -static affect only one library that follows immediately or does it affect all libraries provided on command line?
g++ -static -lutils1 -lutils2
GGC's -static linkage option prevents linkage with shared libraries. So
all libraries that the linkage requires must be static. The
linker must be able to find a static library to resolve all -lname options that
are passed, as well as static versions of all default libraries that GCC silently
appends to the linkage.
That is the intended use of the -static option, although it is possible to make it more flexible.
GCC's -static option works simply by causing GCC to pass the option -static|-Bstatic to the linker (ld), at
a position in the generated ld commandline that precedes all the libraries in
the linkage.
The linker's -static option has a different meaning from GCC's. From the ld manual:
-Bstatic
-dn
-non_shared
-static
Do not link against shared libraries. This is only meaningful on platforms for which shared libraries are supported.
he different variants of this option are for compatibility with various systems.
You may use this option multiple times on the command line: it affects library searching for -l options which follow it.
(My emphasis)
So the linker's -static|-Bstatic option means: Do not link any shared libraries until further notice. At a later
point in the ld commandline you may cancel the effect of -static|-BStatic with the option -Bdynamic, which
will allow dynamic libraries to be linked again, from that point on, until further notice.
Unlike the linker, GCC has no option that cancels the effect of its -static option. However, GCC allows you to pass abitrary options through to ld, via -Wl,<ld-options>.
So you can in fact cancel GCC's -static option at a later point in the commandline like this:
gcc -static -o prog main.o -lfoo -lbar -Wl,-Bdynamic -lgum ...
This will generate a linkage commandline in which -lfoo and -lbar must be resolved to
static libraries but -lgum and any subsequent library can be resolved to
a shared library or a static library, as usual. Though if you wanted to "turn on/turn off"
dynamic linkage like this at different points in the commandline, it would be more
natural not to use GCC's -static option and instead write the equivalent:
gcc -o prog main.o -Wl,-Bstatic -lfoo -lbar -Wl,-Bdynamic -lgum ...
I have 3 classes - I denote those by firstClass,secondClass,thirdClass.
My headers - firstClass.h, secondClass.h, thirdClass.h and sources firstClass.cpp, secondClass.cpp, thirdClass.cpp.
In class thirdClass I create instance of firstClass and two instance of secondClass.
In main.cpp I deamonize and create instance thirdClass.
I want to create static library of thirdclass and linking to main.cpp.
firstClass and thirdClass used the same library libm.a
I created library step by step as following:
g++ -c -I-/usr/include/ -I-/usr/lib/ -I-/home/projects/Learninig firstClass.cpp -lstdc++ -lm-o WsChannel.o -w -m32
g++ -c -I-/usr/include/ -I-/usr/lib/ --I-/home/projects/Learninig secondClass -lstdc++ -o secondClass.o -w -m32
g++ -c -I-/usr/include/ -I-/usr/lib/ --I-/home/projects/Learninig thirdClass.cpp -lstdc++ -lm -o thirdClass.o -w -m32
ar rcs libLearning.a firstClass.o secondClass.o thirdClass.o
g++ main.cpp -L. -lLearning -lm -o MnLearning.o -m32
Compiling was maked correctly without any errors, but when I execute program I have same error. I spent some hours on checking code, but I don't find bugs. So then maybe compiling was incorrect. I did this using some tutorial in web.
If whatever was unclearly I am ready to more explain my question.
Edit: My error:
segfault at 557400000045 ip 00005574bd509dcd sp 00007ffd9e887900 error 4 in MnLearning[5574bd4f2000+26000]
The error is surely inside your own source code. Avoid undefined behavior in it, and be scared of UB.
Your use of -I- is strange, and probably wrong. I recommend removing it (and also, at first, remove the -m32 flag if your computer and distribution is 64 bits; work first to have your program run correctly on your laptop, then port it later to 32 bits Linux by adding the -m32 flag). You might use preprocessor options like -H to be shown what files are included.
I recommend building your library and your program with some build automation tool, such as GNU make or ninja.
Configure your build to compile with all warnings and debug info, i.e. using g++ -Wall -Wextra -g with GCC. Improve your source code to get no warnings. Then use the gdb debugger to understand the behavior of your program (and library).
So then maybe compiling was incorrect.
No, the compiler is probably good, and you should trust it.
The bug is very likely to be in your own code.
My error: segfault at 557400000045 ip 00005574bd509dcd sp 00007ffd9e887900 error 4 in MnLearning[5574bd4f2000+26000]
Segmentation fault is a symptom of some error in your own code (e.g. some buffer overflow, some bad pointer dereference, etc; or other kind of UB).
You might also use valgrind.
I spent some hours on checking code, but I don't find bugs.
You did not spend enough time (some bugs may take you weeks of work to be found), and you forgot to use the debugger, a very handy tool to help you understand the behavior of your program and find bugs in it. Be aware that programming is difficult, and don't be discouraged.
This question already has answers here:
Use both static and dynamically linked libraries in gcc
(3 answers)
Closed 5 years ago.
First of all I would like to give some background information to avoid the XY Problem.
I am trying to compile a c++ program using makefiles and g++. I also have to build any dependencies statically into the program, but not the 'system libraries' (libz.so, libdl.so, libstdc++.so, libm.so, libpthread.so, libc.so and libgcc.so).
To achieve this I specify -static as linker flag, then all dependencies that have to be statically linked and then I use the -Wl, -Bdynamic option, that should tell the linker to link every library, after this option, to be linked in dynamically including the 'system libraries' because they get linked last.(please correct me if I am wrong.)
LDFLAGS += -Lpath/to/dependencies
# These libs should be linked statically
LDFLAGS += -static
LDFLAGS += -llib1
LDFLAGS += -llib2
LDFLAGS += -llib3
# Libs after this should be linked dynamically.
LDFLAGS += -Wl, -Bdynamic
LDFLAGS += -lz # If i dont specify these three libraries (z, pthread, dl)
LDFLAGS += -lpthread # I get undefined reference errors
LDFLAGS += -ldl
When I call make, the program compiles and links just fine, but when I try to execute it I get the error: bash: ./program: No such file or directory.
But the file DOES exist.
When I remove the -static flag from the linker, the program can be excecuted just fine, but the dependencies are linked dynamically, which is not what I want :(.
So when I call file on the program, that was made with the -static flag, I get this:
program: ELF 32-bit LSB executable, Intel 80386, version 1 (GNU/Linux), dynamically linked, interpreter /usr/lib/libc.so.1, for GNU/Linux 4.9.0, not stripped
The problem seems to be that the interpreter is set to /usr/lib/libc.so.1 and not to /lib/ld-linux.so.2 as it should usually be. At least it is, when I compile without the -static option.
What I found out is that the 'interpreter' is in reality the shared library loader and from what I read, I now assume that the bash tells me it can not find the program because the library loader is just wrong (Even though I don't quite understand the details of this).
So basically my question is: Why does the library loader get set to libc.so when I specify the -static option to the linker and how can I tell the linker to use the correct library loader when -static is specified?
Your error is mixing -static and -Bdynamic as both compiler and linker flags. Don't do that. If you use -Wl, gcc just blindly pass these flags to the linker, but if you don't, it rearranges the entire link line. (check what it does with gcc -v).
The mix creates an inconsistent and erroneous link command. I have no idea why gcc doesn't at least warn about that, but it doesn't, and silently sets the dynamic loader to a non-existent file.
You want to use -Wl,-Bstatic and -Wl,-Bdynamic consistently throughout. Not -Bstatic and -Bdynamic, as gcc logic is different from that of ld.
This will create a correct dynamically linked executable with some static libs linked in.
I've been programming on a 32 bit machine, until recently, I upgraded to a 64 bit one. I'm using the latest version of DMD (amd64), on xubuntu 16.04 (amd64).
Before the upgrade, I could easily compile shared libs using dmd -shared 'FILES', but now, it gives an error.
I have a file named q.d:
module q;
export extern(C) int abcd(){
return 4;
}
And now when I do dmd -shared 'q.d', I get this:
nafees#OptiPlex-755:~/Desktop/temp$ dmd -shared q.d
/usr/bin/ld: q.o: relocation R_X86_64_32 against `__dmd_personality_v0' can not be used when making a shared object; recompile with -fPIC
q.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
--- errorlevel 1
and when I do dmd -shared -fPIC q.d:
nafees#OptiPlex-755:~/Desktop/temp$ dmd -shared -fPIC q.d
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libphobos2.a(exception_224_3b4.o): relocation R_X86_64_32 against `__dmd_personality_v0' can not be used when making a shared object; recompile with -fPIC
/usr/lib/x86_64-linux-gnu/libphobos2.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
--- errorlevel 1
How can I get it to compile?
EDIT: The library compiles fine if I use the -m32 flag.
Oh, I just realized I know this problem, sorry it took me so long to realize it though.
You just need to compile against the shared lib Phobos as well to make the shared lib on 64 bit.
dmd -shared q -m64 -fPIC -defaultlib=libphobos2.so
The -defaultlib switch tells it to use an alternate library. By specifying the .so (as opposed to the default static link with a .a file), it uses the shared lib - which happens to be compiled with -fPIC too, so it is all compatible.
Among other advantages here is that one runtime can be shared across all the shared objects and D executables, which means a lot of things just work when you distribute them all (though note you may also need to compile the program that loads this so with the -defaultlib switch too). On 32 bit, the library isn't built with these options regardless... but the result is you can see link errors for multiple definitions in some circumstances.
The one thing to be careful though is that libphobos2.so file is now a runtime dependency too, be sure to distribute it with your own library builds together. You might need to set the LD_LIBRARY_PATH or install it globally for the program to start up correctly, just like any other library (and you might want to version it too btw)
This question already has answers here:
How to link to a shared library without lib* prefix in a different directory? [duplicate]
(2 answers)
Closed 2 years ago.
I compiled a few libraries using Eclipse-CDT on windows. However, when I tried to compile them under linux gcc keeps giving me the error /usr/bin/ld: cannot find -lrequestedLib. I'm using the exact same build settings between the two environments (namely I made sure to add the directories that contain the libraries i need to link to). I'm sure the system has read access rights to the files as well. I'm not sure what to make of this. Please help.
Edit: These are the commands that ecplise runs to build the library:
gcc -I/home/me/lib/ -O3 -Wall -c -fmessage-length=0 -olibToMake.o ../libToMake.c
gcc -L/home/me/lib/ -shared -olibToMake.so libToMake.o -lrequestedLib
Edit 2: The command that renders the error is the second of the two, resulting in the /usr/bin/ld: cannot find -lrequestedLib being output.
Edit 3: I've confirmed that requestedLib.so is a x86_64 binary.
If you don't want to pass -L command line options to gcc(1), be sure to add the path containing the libraries to /etc/ld.so.conf or /etc/ld.so.cond.d/<something>.
Once you've installed your libraries you also need to run ldconfig(8) by hand. (Most new users forget this step because typical package managers take care of this for you when installing new libraries.)
gcc -I/home/me/lib/ -O3 -Wall -c -fmessage-length=0 -olibToMake.o ../libToMake.c
gcc -L/home/me/lib/ -shared -olibToMake.so libToMake.o -lrequestedLib
When building 64-bit shared libraries on x86_64, the -fPIC flag is usually required, or you get a recompile with -fPIC error at shared library link time.
Since you didn't use -fPIC, yet your link succeeded, you are likely using (non-default) gcc that targets i*86 (that is, produces 32-bit output). You can confirm that by running file libToMake.so.
You didn't show that command that actually fails (the one that produces cannot find -lrequestedLib error). I am guessing that that command is using a different gcc (default one?), that targets x86_64. If it looks something like
gcc main.c -L/home/me/lib -lrequestedLib
that command will ignore /home/me/lib/librequestedLib.so (because you can't link together 32-bit and 64-bit code), and will continue searching for librequestedLib elsewhere. When it can't find a 64-bit version of librequestedLib, it will produce the error message you are gettiing.