Where are GDB symbols coming from? - linux

When I load Fedora 28's /usr/bin/ls file into GDB, I can access to the symbol abformat_init, even if it is not present as a string nor in the symbols table of the binary file.
$ file /usr/bin/ls
/usr/bin/ls: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=d6d0ea6be508665f5586e90a30819d090710842f, stripped, too many notes (256)
$ readelf -S /usr/bin/ls | grep abformat
$ nm /usr/bin/ls
nm: /usr/bin/ls: no symbols
$ strings /usr/bin/ls | grep abformat
$ gdb /usr/bin/ls
[...]
Reading symbols from /usr/bin/ls...Reading symbols from /usr/bin/ls...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Missing separate debuginfos, use: dnf debuginfo-install coreutils-8.29-7.fc28.x86_64
(gdb) info symbol abformat_init
abformat_init in section .text of /usr/bin/ls
Where does this symbol comes from? Is there a program that allows to extract them outside of GDB?

TL;DR:
There is a special .gnu_debugdata compressed section in Fedora binaries that GDB reads, and which contains mini-symbols.
Contents of that section can be conveniently printed with eu-readelf -Ws --elf-section /usr/bin/ls
readelf -S /usr/bin/ls | grep abformat
That command is dumping sections. You want symbols instead:
readelf -s /usr/bin/ls | grep abformat
readelf --all /usr/bin/ls | grep abformat
strings /usr/bin/ls | grep abformat
Strings tries to guess what you want, and doesn't output all strings found in the binary. See this blog post and try:
strings -a /usr/bin/ls | grep abformat
Update: I confirmed the results you've observed: abformat does not appear anywhere, yet GDB knows about it.
Turns out, there is a .gnu_debugdata compressed section (described here), which has mini-symbols.
To extract this data, normally you would do:
objcopy -O binary -j .gnu_debugdata /usr/bin/ls ls.mini.xz
However, that is broken on my system (produces empty output), so instead I used dd:
# You may need to adjust the numbers below from "readelf -WS /usr/bin/ls"
dd if=/usr/bin/ls of=ls.mini.xz bs=1 skip=151896 count=3764
xz -d ls.mini.xz
nm ls.mini | grep abformat
This produced:
00000000000005db0 t abformat_init
QED.
Additional info:
Confusing GDB no debugging symbols is addressed in this bug.
objcopy refusing to copy .gnu_debugdata is the subject of this bug.
There is a tool that can conveniently dump this info:
eu-readelf -Ws --elf-section /usr/bin/ls | grep abformat
37: 0000000000005db0 593 FUNC LOCAL DEFAULT 14 abformat_init

Is there a program that allows to extract them outside of GDB?
Yes, you can use nm to extract the symbol, but you should look for the symbol in a separate debug info file, because the binary itself is stripped.
You can use readelf or objdump to know separate debug info file name, see How to know the name and/or path of the debug symbol file which is linked to a binary executable?:
$ objdump -s -j .gnu_debuglink /usr/bin/ls
/usr/bin/ls: file format elf64-x86-64
Contents of section .gnu_debuglink:
0000 6c732d38 2e33302d 362e6663 32392e78 ls-8.30-6.fc29.x
0010 38365f36 342e6465 62756700 5cddcc98 86_64.debug.\...
On Fedora 29 the separate debug info file name for /usr/bin/ls is ls-8.30-6.fc29.x86_64.debug.
Normally, on Fedora, separate debug info is installed to /usr/lib/debug/ directory so the full path to debug info file is /usr/lib/debug/usr/bin/ls-8.30-6.fc29.x86_64.debug.
Now you can look for the symbol with nm:
$ nm /usr/lib/debug/usr/bin/ls-8.30-6.fc29.x86_64.debug | grep abformat_init
0000000000006d70 t abformat_init
Note that separate debug info should be installed with debuginfo-install, this is what gdb is telling you.

Related

Why does rpmbuild rewrite elf headers and how do I stop it?

I have a stripped down spec file that looks like:
Name: some-binary
Version: 6.1.0
Release: 1
License: proprietary
Summary: a binary
%description
Whatever
%install
install -d %{buildroot}/fromrpm
cp %{_sourcedir}/thebinary %{buildroot}//fromrpm/thebinary
%files
/fromrpm/thebinary
Then I run:
$ cd ~/rpmbuild/
$ rpmbuild -bb SPECS/some-binary.spec 2>/dev/null
Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.gxRpSj
Processing files: some-binary-6.1.0-1.x86_64
Provides: some-binary = 6.1.0-1 some-binary(x86-64) = 6.1.0-1
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/dspeyer/rpmbuild/BUILDROOT/some-binary-6.1.0-1.x86_64
Wrote: /home/dspeyer/rpmbuild/RPMS/x86_64/some-binary-6.1.0-1.x86_64.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.WR3Utx
$ rpm2cpio RPMS/x86_64/some-binary-6.1.0-1.x86_64.rpm | cpio -idmv
./fromrpm/thebinary
41456 blocks
$ readelf -a SOURCES/thebinary | grep rela\\.
[ 5] .rela.plt RELA 0000000000400338 00000338
00 .note.gnu.property .note.gnu.build-id .note.ABI-tag .note.go.buildid .rela.plt
Relocation section '.rela.plt' at offset 0x338 contains 29 entries:
$ readelf -a fromrpm/thebinary | grep rela\\.
[ 6] .rela.got.plt RELA 0000000000000000 0143d048
Relocation section '.rela.got.plt' at offset 0x143d048 contains 29 entries:
I'm not actually sure what the rela tags do, but I strongly suspect they're the reason the post-rpm binary when run on rhel 7.9 immediately crashes trying to crash with "Unexpected reloc type in static binary". Yes, the binary is statically linked. (And, yes, I did say "crashes trying to crash": sigsegv inside __libc_fatal)
Any ideas what rpmbuild is trying to do, or how to stop it?

bash: ./helloworld_s: no such file or directory. The file is clearly there

I'm not unfamiliar with bash, but this is the first time I have ever seen this happen.
[OP#localhost linking]$ ls
helloworld-lib.o helloworld-lib.s helloworld_s
[OP#localhost linking]$ ./helloworld_s
bash: ./helloworld_s: No such file or directory
This error occurred while I was testing the linker, ld. The contents of helloworld-lib.s are:
[OP#localhost linking]$ cat helloworld-lib.s
.section .data
helloworld:
.ascii "Hello, world!\n\0"
.section .text
.globl _start
_start:
mov $helloworld, %rdi
call printf
mov $0, %rdi
call exit
This file helloworld_s was produced as follows.
[OP#localhost linking]$ as helloworld-lib.s -o helloworld-lib.o
[OP#localhost linking]$ ld -lc helloworld-lib.o -o helloworld_s
IDK if any of this information is relevant. As an FYI, if I attempt to run the other files, I just get a permission denied (as expected). Any ideas?
EDIT: as suggested, here is the output of ls -l:
[OP#localhost linking]$ ls -l
total 88
-rw-rw-r--. 1 OP OP 968 Mar 23 18:40 helloworld-lib.o
-rw-rw-r--. 1 OP OP 159 Mar 23 18:40 helloworld-lib.s
-rwxrwxr-x. 1 OP OP 14384 Mar 23 18:41 helloworld_s
here is the output of id:
[OP#localhost linking]$ id
uid=1000(OP) gid=1000(OP) groups=1000(OP),10(wheel) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
EDIT: for answer, see comments. See here
As explained in redhat bug #868662 , the recommanded way to link is to let gcc call ld like below;
> gcc -nostartfiles helloworld-lib.o -o helloworld_s -lc
Which results in correct linking;
> ldd helloworld_s
linux-vdso.so.1 => (0x00007ffd283bf000)
libc.so.6 => /lib64/libc.so.6 (0x00007fd011b62000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd011f2f000)
And execution goes fine;
> ./helloworld_s
Hello, world!
Why does ld link to /lib/ld64.so.1 which does not exist ?
Because this is the default setup for a generic system, not only Linux.
Existent executables may be confusingly reported as missing under circumstances where the actual issue is that they cannot be executed.
Actual causes vary, but include things such as
the file is defective, perhaps as a result of invalid linking as mentioned in another answer
the file is for a different architecture or ABI unsupported by the platform
the file lacks an execute permission bit for the user attempting to do so
the file is on a volume mounted with flags which prohibit execution
In many of these cases, it's clear that a more specific and relevant error message would have been preferable, however, sometimes what is actually implemented (or triggered by less than obvious paths of failure) can indeed be confusing in the sense of labelling something that is "unusable" as being "missing" How precise errors are can vary somewhat between environments.

"No such file or directory" error when running a dynamically-linked ARM executable

I am cross-compiling for an ARM embedded Linux platform running Ubuntu 15.04. Previous, statically-linked, versions of this program have run just fine. Recently I needed to link in libproprietary (lib name changed for this question), which is only available as an .so.
libproprietary.so works; I am able to run a different program that depends on it.
$ uname -a
Linux <hostname> 3.10.92-71 #1 SMP PREEMPT Fri Dec 18 00:38:54 BRST 2015 armv7l armv7l armv7l GNU/Linux
The problem:
$ ./myprogram
-bash: ./myprogram: No such file or directory
What I've tried:
$ ldd ./myprogram
libproprietary.so.2 => /usr/lib/libproprietary.so.2 (0xb6bbc000)
librt.so.1 => /lib/arm-linux-gnueabihf/librt.so.1 (0xb6ba6000)
libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0xb6b92000)
libusb-1.0.so.0 => /lib/arm-linux-gnueabihf/libusb-1.0.so.0 (0xb6b72000)
libstdc++.so.6 => /usr/lib/arm-linux-gnueabihf/libstdc++.so.6 (0xb6ab8000)
libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0xb6a43000)
libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0xb6955000)
/usr/lib/ld.so.1 => /lib/ld-linux-armhf.so.3 (0xb6f28000)
libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0xb692c000)
libudev.so.1 => /lib/arm-linux-gnueabihf/libudev.so.1 (0xb690f000)
$ file ./myprogram
./myprogram: ELF 32-bit LSB executable, ARM, EABI5 version 1 (GNU/Linux), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=881d76b2ce20f32aef95796b4fee9f01e492a7d2, not stripped
$ file /bin/ls
/bin/ls: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=a687c2baf9963c62c6abd209863d360dd0863686, stripped
$ readelf -l myprogram| grep interpreter
[Requesting program interpreter: /usr/lib/ld.so.1]
$ strace ./myprogram
execve("./myprogram", ["./myprogram"...], [/* 21 vars */]) = -1 ENOENT (No such file or directory)
write(2, "strace: exec: No such file or di"..., 40strace: exec: No such file or directory
) = 40
exit_group(1) = ?
+++ exited with 1 +++
$ ls -l myprogram
-rwxrwxr-x [details snipped]
$ file /usr/lib/libproprietary.so.<version>
/usr/lib/libproprietary.so.<version>: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=d9b4ba3b9ec03779792984bc8a89ceede5737455, stripped
It appears that the interpreter, /usr/lib/ld.so.1, does not exist. I'm going to guess that it should be similar to what /bin/ls is using:
$ readelf -l /bin/ls | grep interpreter
[Requesting program interpreter: /lib/ld-linux-armhf.so.3]
I don't know how to fix this in my link step however.
ccache arm-linux-gnueabihf-g++ -o myprogram -static -Wl,--whole-archive -lpthread -Wl,--no-whole-archive -Wl,-no-undefined -pthread -Wl,-Bdynamic -lproprietary -Wl,-rpath-link=result/debug/lib -Wl,-rpath=\$ORIGIN/../../../../result/debug/lib variant-dir/debug/core-.o -Lresult/debug/lib -Wl,-Bstatic -l<other libs>
I did try moving -Wl,-Bdynamic -lproprietary after the core-.o word, but that didn't change anything.
It appears that the interpreter, /usr/lib/ld.so.1, does not exist. I'm going to guess that it should be similar to what /bin/ls is using:
Correct. You could change the interpreter by adding -Wl,--dynamic-linker=/lib/ld-linux-armhf.so.3 flag at link time.
However,
Your toolchain may be targeting a different version of libc from the one you actually have installed (in which case the program will likely crash).
If it is targeting correct libc, then your toolchain is mis-configured.

How to make profilers (valgrind, perf, pprof) pick up / use local version of library with debugging symbols when using mpirun?

Edit: added important note that it is about debugging MPI application
System installed shared library doesn't have debugging symbols:
$ readelf -S /usr/lib64/libfftw3.so | grep debug
$
I have therefore compiled and instaled in my home directory my owne version, with debugging enabled (--with-debug CFLAGS=-g):
$ $ readelf -S ~/lib64/libfftw3.so | grep debug
[26] .debug_aranges PROGBITS 0000000000000000 001d3902
[27] .debug_pubnames PROGBITS 0000000000000000 001d8552
[28] .debug_info PROGBITS 0000000000000000 001ddebd
[29] .debug_abbrev PROGBITS 0000000000000000 003e221c
[30] .debug_line PROGBITS 0000000000000000 00414306
[31] .debug_str PROGBITS 0000000000000000 0044aa23
[32] .debug_loc PROGBITS 0000000000000000 004514de
[33] .debug_ranges PROGBITS 0000000000000000 0046bc82
I have set both LD_LIBRARY_PATH and LD_RUN_PATH to include ~/lib64 first, and ldd program confirms that local version of library should be used:
$ ldd a.out | grep fftw
libfftw3.so.3 => /home/narebski/lib64/libfftw3.so.3 (0x00007f2ed9a98000)
The program in question is parallel numerical application using MPI (Message Passing Interface). Therefore to run this application one must use mpirun wrapper (e.g. mpirun -np 1 valgrind --tool=callgrind ./a.out). I use OpenMPI implementation.
Nevertheless, various profilers: callgrind tool in Valgrind, CPU profiling google-perfutils and perf doesn't find those debugging symbols, resulting in more or less useless output:
calgrind:
$ callgrind_annotate --include=~/prog/src --inclusive=no --tree=none
[...]
--------------------------------------------------------------------------------
Ir file:function
--------------------------------------------------------------------------------
32,765,904,336 ???:0x000000000014e500 [/usr/lib64/libfftw3.so.3.2.4]
31,342,886,912 /home/narebski/prog/src/nonlinearity.F90:__nonlinearity_MOD_calc_nonlinearity_kxky [/home/narebski/prog/bin/a.out]
30,288,261,120 /home/narebski/gene11/src/axpy.F90:__axpy_MOD_axpy_ij [/home/narebski/prog/bin/a.out]
23,429,390,736 ???:0x00000000000fc5e0 [/usr/lib64/libfftw3.so.3.2.4]
17,851,018,186 ???:0x00000000000fdb80 [/usr/lib64/libmpi.so.1.0.1]
google-perftools:
$ pprof --text a.out prog.prof
Total: 8401 samples
842 10.0% 10.0% 842 10.0% 00007f200522d5f0
619 7.4% 17.4% 5025 59.8% calc_nonlinearity_kxky
517 6.2% 23.5% 517 6.2% axpy_ij
427 5.1% 28.6% 3156 37.6% nl_to_direct_xy
307 3.7% 32.3% 1234 14.7% nl_to_fourier_xy_1d
perf events:
$ perf report --sort comm,dso,symbol
# Events: 80K cycles
#
# Overhead Command Shared Object Symbol
# ........ ....... .................... ............................................
#
32.42% a.out libfftw3.so.3.2.4 [.] fdc4c
16.25% a.out 7fddcd97bb22 [.] 7fddcd97bb22
7.51% a.out libatlas.so.0.0.0 [.] ATL_dcopy_xp1yp1aXbX
6.98% a.out a.out [.] __nonlinearity_MOD_calc_nonlinearity_kxky
5.82% a.out a.out [.] __axpy_MOD_axpy_ij
Edit Added 11-07-2011:
I don't know if it is important, but:
$ file /usr/lib64/libfftw3.so.3.2.4
/usr/lib64/libfftw3.so.3.2.4: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, stripped
and
$ file ~/lib64/libfftw3.so.3.2.4
/home/narebski/lib64/libfftw3.so.3.2.4: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, not stripped
If /usr/lib64/libfftw3.so.3.2.4 is listed in callgrind output, then your LD_LIBRARY_PATH=~/lib64 had no effect.
Try again with export LD_LIBRARY_PATH=$HOME/lib64. Also watch out for any shell scripts you invoke, which might reset your environment.
You and Employed Russian are almost certainly right; the mpirun script is messing things up here. Two options:
Most x86 MPI implementations, as a practical matter, treat just running the executable
./a.out
the same as
mpirun -np 1 ./a.out.
They don't have to do this, but OpenMPI certainly does, as does MPICH2 and IntelMPI. So if you can do the debug serially, you should just be able to
valgrind --tool=callgrind ./a.out.
However, if you do want to run with mpirun, the issue is probably that your ~/.bashrc
(or whatever) is being sourced, undoing your changes to LD_LIBRARY_PATH etc. Easiest is just to temporarily put your changed environment variables in your ~/.bashrc for the duration of the run.
The way recent profiling tools typically handle this situation is to consult an external, matching non-stripped version of the library.
On debian-based Linux distros this is typically done by installing the -dbg suffixed version of a package; on Redhat-based they are named -debuginfo.
In the case of the tools you mentioned above; they will typically Just Work (tm) and find the debug symbols for a library if the debug info package has been installed in the standard location.

How do I find out what all symbols are exported from a shared object?

I have a shared object (dll). How do I find out what all symbols are exported from that?
Do you have a "shared object" (usually a shared library on AIX), a UNIX shared library, or a Windows DLL? These are all different things, and your question conflates them all :-(
For an AIX shared object, use dump -Tv /path/to/foo.o.
For an ELF shared library, use readelf -Ws --dyn-syms /path/to/libfoo.so, or (if you have GNU nm) nm -D /path/to/libfoo.so.
For a non-ELF UNIX shared library, please state which UNIX you are interested in.
For a Windows DLL, use dumpbin /EXPORTS foo.dll.
objdump is another good one on linux.
If it is a Windows DLL file and your OS is Linux then use winedump:
$ winedump -j export pcre.dll
Contents of pcre.dll: 229888 bytes
Exports table:
Name: pcre.dll
Characteristics: 00000000
TimeDateStamp: 53BBA519 Tue Jul 8 10:00:25 2014
Version: 0.00
Ordinal base: 1
# of functions: 31
# of Names: 31
Addresses of functions: 000375C8
Addresses of name ordinals: 000376C0
Addresses of names: 00037644
Entry Pt Ordn Name
0001FDA0 1 pcre_assign_jit_stack
000380B8 2 pcre_callout
00009030 3 pcre_compile
...
On *nix check nm. On windows use the program Dependency Walker
see man nm
GNU nm lists the symbols from object files objfile.... If no object
files are listed as arguments, nm assumes the file a.out.
Use: nm --demangle <libname>.so
The cross-platform way (not only cross-platform itself, but also working, at the very least, with both *.so and *.dll) is using reverse-engineering framework radare2. E.g.:
$ rabin2 -s glew32.dll | head -n 5
[Symbols]
vaddr=0x62afda8d paddr=0x0005ba8d ord=000 fwd=NONE sz=0 bind=GLOBAL type=FUNC name=glew32.dll___GLEW_3DFX_multisample
vaddr=0x62afda8e paddr=0x0005ba8e ord=001 fwd=NONE sz=0 bind=GLOBAL type=FUNC name=glew32.dll___GLEW_3DFX_tbuffer
vaddr=0x62afda8f paddr=0x0005ba8f ord=002 fwd=NONE sz=0 bind=GLOBAL type=FUNC name=glew32.dll___GLEW_3DFX_texture_compression_FXT1
vaddr=0x62afdab8 paddr=0x0005bab8 ord=003 fwd=NONE sz=0 bind=GLOBAL type=FUNC name=glew32.dll___GLEW_AMD_blend_minmax_factor
As a bonus, rabin2 recognizes C++ name mangling, for example (and also with .so file):
$ rabin2 -s /usr/lib/libabw-0.1.so.1.0.1 | head -n 5
[Symbols]
vaddr=0x00027590 paddr=0x00027590 ord=124 fwd=NONE sz=430 bind=GLOBAL type=FUNC name=libabw::AbiDocument::isFileFormatSupported
vaddr=0x0000a730 paddr=0x0000a730 ord=125 fwd=NONE sz=58 bind=UNKNOWN type=FUNC name=boost::exception::~exception
vaddr=0x00232680 paddr=0x00032680 ord=126 fwd=NONE sz=16 bind=UNKNOWN type=OBJECT name=typeinfoforboost::exception_detail::clone_base
vaddr=0x00027740 paddr=0x00027740 ord=127 fwd=NONE sz=235 bind=GLOBAL type=FUNC name=libabw::AbiDocument::parse
Works with object files too:
$ g++ test.cpp -c -o a.o
$ rabin2 -s a.o | head -n 5
Warning: Cannot initialize program headers
Warning: Cannot initialize dynamic strings
Warning: Cannot initialize dynamic section
[Symbols]
vaddr=0x08000149 paddr=0x00000149 ord=006 fwd=NONE sz=1 bind=LOCAL type=OBJECT name=std::piecewise_construct
vaddr=0x08000149 paddr=0x00000149 ord=007 fwd=NONE sz=1 bind=LOCAL type=OBJECT name=std::__ioinit
vaddr=0x080000eb paddr=0x000000eb ord=017 fwd=NONE sz=73 bind=LOCAL type=FUNC name=__static_initialization_and_destruction_0
vaddr=0x08000134 paddr=0x00000134 ord=018 fwd=NONE sz=21 bind=LOCAL type=FUNC name=_GLOBAL__sub_I__Z4funcP6Animal
You can use gnu objdump. objdump -p your.dll. Then pan to the .edata section contents and you'll find the exported functions under [Ordinal/Name Pointer] Table.
Usually, you would also have a header file that you include in your code to access the symbols.

Resources