I have a question: if we have an application, which uses no threads, we can link it in two ways:
1) Link as usual, without -lpthread and -ldl
2) Add to the link two libraries: libpthread and libdl.
E.g.
$ cat a.c
int main(){printf("Hehe");}
$ gcc a.c -w -o a
$ gcc a.c -w -o a1 -ldl -lpthread
By default, both libs are dynamically linked:
$ ldd a
linux-gate.so.1
libc.so.6
/lib/ld-linux.so.2
$ ldd a1
linux-gate.so.1
libdl.so.2
libpthread.so.0
libc.so.6
/lib/ld-linux.so.2
How much difference will be there between version a and version a1 ? What will be working in different way inside application itself and int glibc ? Will linking of pthreads change something inside from thread-unsafe to thread-safe algorithm?
E.g.
$ strace ./a 2>&1 |wc -l
73
$ strace ./a1 2>&1 |wc -l
103
In a1 trace, two additional libs are loaded, some more mprotects are called, and added section of:
set_tid_address; set_robust_list; rt_sigaction x 2; rt_sigprocmask; getrlimit; uname
glibc itself contains stub code for many pthread functions. These glibc pthread functions do nothing. However, when the program is linked with libpthread then those stubs are replaced with the real pthread locking functions.
This is intended for use in libraries that need to be thread safe but do not use threads themselves. These libraries can use pthread locks, but those locks will not actually happen until a program or library that links to libpthread is loaded.
Related
I link a shared library on the command line while building an executable. Running ldd on that executable is not showing the linked shared library.
After looking at some of the output of the linker, I have even tried adding the -Wl,--no-as-needed option and that didn't help either.
foo.c:
#include <stdio.h>
void foo () {
printf ("Hello world\n");
}
main.c:
#include <stdio.h>
int main () {
printf ("In main \n");
foo ();
}
Here's the command I used to compile and link:
$ gcc -Wl,--no-as-needed main.c -o main -L./ -lfoo
/bin/ld: cannot find -lfoo
collect2: error: ld returned 1 exit status
$ gcc -c foo.c -shared -Wl,-soname,libfoo.so -o libfoo.so
$ ls -l libfoo.so
-rw-r--r-- 1 apple eng 1488 Jun 4 04:44 libfoo.so
$ gcc -Wl,--no-as-needed main.c -o main -L./ -lfoo
$ ldd main
linux-vdso.so.1 => (0x00007fffbdd6c000)
libc.so.6 => /lib64/libc.so.6 (0x00007f6367e23000)
/lib64/ld-linux-x86-64.so.2 (0x00005556e268a000)
libfoo.so does not show up above.
$ objdump -x main | grep NEEDED
NEEDED libc.so.6
Why isn't libfoo.so showing up as NEEDED?
By using the gcc option -c you're telling it to create only object from foo.c, so the only product you're getting from this gcc command is an object file. The fact that its suffix is .so is only because you forced it using -o option. If you run this command without -o you'd see that the output is just foo.o - an object file.
If you omit the -c from the gcc command you'd get the shared object you wanted.
Running file on the output file shows the difference (note that I'm not setting the output name using -o and letting gcc use its default names):
With -c:
> gcc -c foo.c -shared
> file foo.o
foo.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
Without -c:
> gcc foo.c -shared
> file a.out
a.out: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=a63581bfc45f845c501ffb6635, not stripped
^^^
|||
This command does not generate a shared library:
gcc -c foo.c -shared -Wl,-soname,libfoo.so -o libfoo.so
It generates an object file libfoo.so that is later statically linked with your code. Proof: remove the lib file, the program will still run.
Solution:
Compile the object file separately, then convert it into a shared library. You will have to tell the loader where to search for the shared libraries by setting LD_LIBRARY_PATH:
gcc -c foo.c
gcc foo.o -shared -o libfoo.so
gcc main.c -o main -L./ -lfoo
export LD_LIBRARY_PATH=`pwd`
ldd ./main
# linux-vdso.so.1 (0x00007ffe0f7cb000)
# libfoo.so => /home/---/tmp/libfoo.so (0x00007f9bab6ec000)
# libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9bab2fb000)
# /lib64/ld-linux-x86-64.so.2 (0x00007f9babaf0000)
it's my first question on stackoverflow, so I will try to do it well.
Context:
I would like to deliver a program who could run on every Linux distribution (for example, a program who will use C++11, running on a system who don't have the C++11 library).
For that I would like to copy all the libraries who are used by my program and put them in a folder with the executable, so it can use these libraries instead of the system's one.
I got 2 environments to test:
- Opensuse, with (GNU libc) 2.19
- Ubuntu, with (Ubuntu EGLIBC 2.17-Oubuntu5.1) 2.17
I compile my program under Opensuse, and run it under Ubuntu. The program works well when it uses the default library.
Project:
Here is the main.c:
int main(int ac, char **av) {
printf("Hello World !\n");
}
Here is the tree of my folder under Opensuse (same under Ubuntu without main.c et exec.sh):
+ project
|
+--- main.c
+--- a.out
+--- exec.sh
+---+ lib
|
+--- libc.so.6
+--- ld-linux-x86-64.so.2
And finally here is the ldd and the readelf when I launch the program with a simple compilation:
> gcc main.c -o a.out
> ldd ./a.out
linux-vdso.so.1 (0x00007fff85f57000)
libc.so.6 => /lib64/libc.so.6 (0x00007f1fdaaaf000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1dae75000)
> readelf -d a.out | grep "library\|Library"
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
I have done some research, and finally found this post who explain a bit about the ld-linux.so.
Here is the script that I use to compile:
#!/bin/bash
dir=`pwd`
execName="a.out"
libDir="/lib"
linker="ld-linux-x86-64.so.2"
gcc main.c -o ${execName} -Wl,--rpath=${dir}${libDir} -Wl,--dynamic-linker=${dir}${libDir}/${linker}
When I launch my a.out compiling with the script on my Opensuse i got the following:
> ./a.out
Hello World !
> ldd ./a.out
linux-vdso.so.1 (0x00007f3222e27000)
libc.so.6 => /path/to/project/lib/libc.so.6 (0x00007f3222a7e000)
/path/to/project/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x000073222e2b000)
> readelf -d a.out | grep "library\|Library"
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000001d (RUNPATH) Library runpath: [/path/to/project/lib]
Problem:
Now, when I launch this executable a.out (compiled under Opensuse) on my Ubuntu, i got the following output:
> ./a.out
./a.out: Permission denied.
> ldd ./a.out
linux-vdso.so.1 (0x00007fff5b5fe000)
libc.so.6 => /path/to/project/lib/libc.so.6 (0x00007f47df480000)
/path/to/project/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007f47df82a000)
> readelf -d a.out | grep "library\|Library"
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000001d (RUNPATH) Library runpath: [/path/to/project/lib]
I keep having this permission denied when I launch the executable, and I don't know why.
I transfer my a.out and my lib folder from Opensuse to Ubuntu with Filezilla, and a.out isn't an executable after that transfer, so I need to do:
chmod 755 a.out
I have the same tree under Opensuse and Ubuntu
The 2 libraries in the lib folder are default library from Opensuse
Any help about the permission denied, or another way to do what I want will be welcome.
Thanks in advance for your help!
Note: I can't use LD_PRELOAD, since you need to be root to use it, it will not work for what I want to do.
Also i would like to avoid the static compilation, but if it's the only solution i will think about it
There are many things that could be wrong with this. Here are a few suggestions.
Check file permissions for copied libraries
You've mentioned that whatever you're using to copy your a.out alters permissions for that file. Libraries need specific permissions, too, so check those.
Check for broken symlinks
Many of those libraries are typically actually symlinks to the real binary. If you've only copied the links and not the underlying binary, it won't work.
Check for mismatched libraries
As you know, libc is more than just a single file. For example, on my Linux machine, if I run ldd /lib64/libc.so.6 I get this result:
/lib64/ld-linux-x86-64.so.2 (0x0000003531200000)
linux-vdso.so.1 => (0x00007ffe2c78c000)
Unless you are also copying all of the libraries that are needed, it won't work.
Consider using -R for the linker
The -L flag tells the linker where it can find the libraries, but there's also a -R flag that tells the resulting executable where to find the libraries at runtime. It's not clear to me whether you'll need that or not.
You can follow the easier way that is compiling your application not depending on share object. In this case, your application will be static compiled, and you should use the flag -static.
Take a look at my example:
$ cat main.c
#include <stdio.h>
int main(int ac, char **av) {
printf("Hello World !\n");
}
$ gcc main.c -o main -static
$ ldd main
not a dynamic executable
Reproduce the permission error by running the application under strace utility and analyze the logs to isolate the real runtime issue.
$ strace -fv ./a.out
Any help about the permission denied
That part is simple: not only the a.out must be made executable with chmod, but the /path/to/project/lib/ld-linux-x86-64.so.2 must be executable as well.
Note: I can't use LD_PRELOAD, since you need to be root to use it
You are mistaken: you absolutely do not need to be root to use LD_PRELOAD (unless you are using it on set-uid binary).
This is my system:
# cat /etc/SuSE-release
openSUSE 11.4 (i586)
VERSION = 11.4
CODENAME = Celadon
# uname -r
2.6.37.1-1.2-default
# gcc --version
gcc (SUSE Linux) 4.5.1 20101208 [gcc-4_5-branch revision 167585]
...
Now I have these files:
foo.h:
void foo();
foo.c:
#include <stdio.h>
void foo(){
printf("Hello World\n");
}
main.c:
#include <foo.h>
int main(){
foo();
return 0;
}
...and I do the following:
# gcc -c -fpic foo.c
# gcc -shared -o libfoo.so foo.o
I then copy foo.h to /usr/include and libfoo.so to /lib, (note that there is no ldconfig run) then compile with:
# gcc -o test main.c -l foo
The problem:
Without running ldconfig, meaning ld.so.cache doesn't have libfoo.so. Yet, # ./test ran just fine (!?). Somehow ld is aware and auto-loads my lib. This is the output of ldd:
# ldd test
linux-gate.so.1 => (0xffffe000)
libfoo.so => /lib/libfoo.so (0xb77d1000)
libc.so.6 => /lib/libc.so.6 (0xb7668000)
/lib/ld-linux.so.2 (0xb77e1000)
I've looked around and already crossed out the possibility that RPATH is at work here. (Running readelf -d test yields no RPATH. EDIT: Also, those environment variables you see in ld.so man page (like LD_LIBRARY_PATH) is not set.
Why is this happening? An interesting note is that on a Red Hat system (CentOS 6), everything ran as expected (I have to do a ldconfig to reload the cache)
The Linux man page on ldconfig is unclear as to the real intention of the cache, so I quote the BSD manpage (emphasis mine):
The ldconfig utility is used to prepare a set of "hints" for use by the dynamic linker to facilitate quick lookup of shared libraries available in multiple directories. It scans a set of built-in system directories and any directories specified on the command line (in the given order) looking for shared libraries and stores the results in a system file to forestall the overhead that would otherwise result from the directory search operations the dynamic linker would have to perform to load the required shared libraries.
It is not required to run ldconfig. It is only supposed to speed things up. The runtime linker knows it needs to look in /usr/lib, but it will actually only do this after it can't locate the lib in its cache.
In Fedora 18, I wrote a file "a.c" like below:
#include <stdio.h>
void main(void)
{
printf("a");
}
and then I compiled it with gcc 4.7.2 : gcc a.c -o a
at last used ldd 2.16: ldd -u ./a
The output is so strange:
Unused direct dependencies:
/lib64/libc.so.6
I think the object file really use /lib64/libc.so.6 because of printf(). And in Red Hat 6,
there is no this strange appearance.
just ldd -u doesn't work too well; you are advised to use ldd -u -r.
See e.g. https://bugzilla.redhat.com/show_bug.cgi?id=216864
At my company we have a really annoying problem with our linker (ld 2.17). It links very very slow on a relatively fast system (Core Duo, 2GB Ram) and I don't really now how to fix this. It takes about five to ten minutes to compile a relatively big project (which takes about 5 seconds to link on my Gentoo system).
Personally i think it is a huge productivity killer, at least for me. We tried to use a more recent version of ld (2.19) but without any success. I asked in #debian on #freenode, but this problem seems to be very unique. I did not find any information about similar problems on the net. It only happens when we build with debug symbols. I changed the gcc debug-information flags to -g, -g3, and -ggdb, but that did not help either.
So my question is, how do you profile and debug a linker? I have never done anything like it, and I am unable to find any documentation about it. Basically any reasonable gprof gmon.out would be very helpful, as I could ask the binutils developers about a concrete problem. I am just completely oblivious about this.
Edit: We 'fixed' our problem switching to debian lenny on most systems. Thanks for the answers!
You can try gold (binutils-gold) instead of ld.
It is supposed to be faster.
Here is a quote from Wikipedia Gold(linker)
The motivation for writing gold was to
make a linker that is faster than the
GNU linker[3], especially for large
applications coded in C++.
The author of gold (Ian Lance Taylor) has published an (longish) article about linkers where he explains his motifs in writing gold and why most linkers are slow. If you are interested in the inner workings of linkers this article is worth reading.
If you're observing the slowdown running gcc (as opposed to directly running the linker as ld), try compiling with
$ gcc -save-temps -v [... rest of your command line ...]
This will print out all the intermediary commands, such as the internal collect2 and final ld, as well as ensure that the objects passed to those commands will remain on disk even after the command has completed.
Then you should be able to run the commands individually to find the worst stage, and then run it with different options or profiling.
For example,
$ echo 'int main() {}' > test.c
$ gcc -save-temps -v test.c
Using built-in specs.
Target: x86_64-pc-linux-gnu
Configured with: /var/tmp/paludis/sys-devel-gcc-4.3.3-r2/work/gcc-4.3.3/configure --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/4.3.3 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.3.3 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.3.3/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.3.3/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/include/g++-v4 --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --disable-altivec --disable-fixed-point --enable-nls --without-included-gettext --with-system-zlib --disable-checking --disable-werror --enable-secureplt --enable-multilib --enable-libmudflap --disable-libssp --enable-libgomp --enable-cld --disable-libgcj --enable-objc-gc --enable-languages=c,c++,objc,obj-c++,treelang,fortran --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --with-bugurl=http://bugs.gentoo.org/ --with-pkgversion='Gentoo 4.3.3-r2 p1.1, pie-10.1.5'
Thread model: posix
gcc version 4.3.3 (Gentoo 4.3.3-r2 p1.1, pie-10.1.5)
COLLECT_GCC_OPTIONS='-save-temps' '-v' '-mtune=generic'
/usr/libexec/gcc/x86_64-pc-linux-gnu/4.3.3/cc1 -E -quiet -v test.c -D_FORTIFY_SOURCE=2 -mtune=generic -fpch-preprocess -o test.i
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../x86_64-pc-linux-gnu/include"
#include "..." search starts here:
#include search starts here:
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/include
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/include-fixed
/usr/include
End of search list.
COLLECT_GCC_OPTIONS='-save-temps' '-v' '-mtune=generic'
/usr/libexec/gcc/x86_64-pc-linux-gnu/4.3.3/cc1 -fpreprocessed test.i -quiet -dumpbase test.c -mtune=generic -auxbase test -version -o test.s
GNU C (Gentoo 4.3.3-r2 p1.1, pie-10.1.5) version 4.3.3 (x86_64-pc-linux-gnu)
compiled by GNU C version 4.3.3, GMP version 4.2.4, MPFR version 2.4.1-p5.
warning: GMP header version 4.2.4 differs from library version 4.3.1.
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 20f3dbffbfd03e5311a257ae1239cd71
COLLECT_GCC_OPTIONS='-save-temps' '-v' '-mtune=generic'
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../x86_64-pc-linux-gnu/bin/as -V -Qy -o test.o test.s
GNU assembler version 2.19.1 (x86_64-pc-linux-gnu) using BFD version (GNU Binutils) 2.19.1
COMPILER_PATH=/usr/libexec/gcc/x86_64-pc-linux-gnu/4.3.3/:/usr/libexec/gcc/x86_64-pc-linux-gnu/4.3.3/:/usr/libexec/gcc/x86_64-pc-linux-gnu/:/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/:/usr/lib/gcc/x86_64-pc-linux-gnu/:/usr/libexec/gcc/x86_64-pc-linux-gnu/4.3.3/:/usr/libexec/gcc/x86_64-pc-linux-gnu/:/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/:/usr/lib/gcc/x86_64-pc-linux-gnu/:/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../x86_64-pc-linux-gnu/bin/
LIBRARY_PATH=/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/:/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/:/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../x86_64-pc-linux-gnu/lib/:/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-save-temps' '-v' '-mtune=generic'
/usr/libexec/gcc/x86_64-pc-linux-gnu/4.3.3/collect2 --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../lib64/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/crtbegin.o -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../x86_64-pc-linux-gnu/lib -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../.. test.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/crtend.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../lib64/crtn.o
$ ls
a.out test.c test.i test.o test.s
$ /usr/libexec/gcc/x86_64-pc-linux-gnu/4.3.3/collect2 --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../lib64/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/crtbegin.o -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../x86_64-pc-linux-gnu/lib -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../.. test.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/crtend.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../lib64/crtn.o
$ .../collect2 -v ...
collect2 version 4.3.3 (x86-64 Linux/ELF)
/usr/bin/ld -v --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../lib64/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/crtbegin.o -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../x86_64-pc-linux-gnu/lib -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../.. test.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/crtend.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../lib64/crtn.o
GNU ld (GNU Binutils) 2.19.1
$
If you need help building a debug version of ld, here's a quick recipe to get you going.
$ sudo apt-get install build-essential dpkg-dev
$ sudo apt-get build-dep binutils
$ apt-get source binutils
$ cd binutils-*
$ DEB_BUILD_OPTIONS='debug noopt nostrip' dpkg-buildpackage -uc -us
$ cd ..
$ sudo dpkg -i *.deb
Instead of that quick hack, though, I'd recommend using schroot or sbuild to avoid polluting your own system.
To answer the profiling question; you should look at OProfile - this is a system-level profiler that can profile multiple running processes. It should allow you to identify which sub-process of the link is taking the most time, and furthermore will show which functions the most time is being spent.
I'd like to suggest two ways to check:
use strace to check what files linker is loading/parsing to link; with this you may know is there any unnecessary path is searched by linker.
use ld with -verbose option to know what ld is doing. Five minutes compare to five seconds should not be the linker's problem, it should be some problem of your host machine or some option.