cross-gdb: fix a mismatch between libthread_db and libpthread - linux

I' ve a cross-gdb configured with --host=i686-pc-linux-gnu --target=powerpc-e300c3-linux-gnu. i can correctly debug an application on a remote board with gdbserver but i get an error about a version mismatch between libthread_db and libpthread so i can't debug threads correctly (gdb recognizes only one thread instead of three threads). Maybe it's is due to a different version of libc: on host machine i've libc2.15 and on target machine lib2.5. I tried to rebuid libc2.5 for host in order to link gdb against to it but it's an hell. Before i get crazy to rebuild it, could someone confirm that it's a libc problem?

I tried to rebuid libc2.5 for host in order to link gdb against to it
That's not what you need.
What you need, is for gdb to find and load libthread_db.so.1, that matches your target libpthread.so.0.
For this, you need to
build libc-2.5 for host, and
set GDB's libthread-db-search-path such that it finds the libthread_db.so.1 built in step 1.
You don't actually need to build the entire libc in step 1. Something like this should suffice:
mkdir build && cd build
../configure --prefix=/usr
make -C ../nptl_db objdir=`pwd`
Update:
i have GDB 6.6 and there isn't libthread-db-search-path. What is another way to specify that path?
That GDB will just dlopen("libthread_db.so.1", ...). So to make it find the right libthread_db.so.1, you need to adjust LD_LIBRARY_PATH. Using bash:
LD_LIBRARY_PATH=/tmp/glibc-2.5/build/nptl_db gdb /path/to/target/a.out

Related

How to make gdb for a target and use it there

I am trying to compile gdb-8.2 from source.
Build machine: x86-64
Host AND target: arm-linux-gnueabi
I ran:
CC=arm-linux-gnueabi-gcc ./configure --host=arm-linux-gnueabi --target=arm-linux-gnueabi
make
Then I ran:
make DESTDIR=<Some Path>/gdb_installation install
So I got a usr folder inside gdb_installation folder. I copied the usr/local/bin/gdb to my target and ran
./gdb
Output:
#./gdb
#
But it does not show anything. It exits without any message.
What am I missing here?
Running the file command shows that the gdb executable is indeed built for my target.
PS: Running a sample hello world program using arm-linux-gnueabi-gcc works perfectly fine on the target; and file command shows the same output that it did for gdb.
What am I missing here?
Your build looks correct, but doesn't work. It's not clear why, so you need to debug that.
What is the exit status of this gdb on the target?
./gdb --version; echo $?
Does it actually do anything? strace ./gdb --version
Is there anything interesting in the kernel message log?
Depending on answers to above questions, further guesses of what has gone wrong will be possible.
Perhaps there is some .gdbinit that tells GDB to quit? What does this do:
./gdb -nx --version?

wxWidgets cross compilation from linux to windows - what does "--host=" mean?

I'm cross compiling wxWidgets using that tutorial. Could anyone explain me what --host=i586-mingw32msvc does?
I know that prefix give the information where cross-compiler is. However, I compiled minGW from source manually, because then I can use the newest GCC so I can't only copy&paste.
Prefix should be prefix=/usr/local/i386-mingw32 because there is my cross-compiler. And what about host?
I use Lubuntu 14.04 LTS AMD64 .
--host is a typical ./configure option that specifies the "host" system for the program to run on, i.e. the configuration that you will be compiling for.
In other words, --host activates cross-compiling behaviour if you specify a host that is not the same as your build machine.
The format of the host spec differs occasionally between platforms, but it generally takes the form
processorfamily-osfamily[-osversion][-abi]

Cross compiling libSDL

I'm trying to cross-compile libSDL version 1.2 for a custom made, debian based Linux system. The toolchain I'm using is already configured properly so that I just run gcc/g++ on my the desired code and the resulting output is compatible with the target machine.
When I run ./configure --help in the libSDL source directory, I see that I can basically just set some environment variables to point to my cross-compiler.
However, I also see the following options:
System types:
--build=BUILD configure for building on BUILD [guessed]
--host=HOST cross-compile to build programs to run on HOST [BUILD]
I looked into the configure.in, build-scripts/config.sub, and build-scripts/config.guess files but couldn't really figure out how it works.
Are these options required? If not, is it a good idea to use them?
With autotools, --build is what you are building on and --host is what you want it to run on (there's also --target, but that's only important if what you're compiling is itself a compiler). Autotools will generally figure out --build on their own, so don't specify it if you don't have to (but look in /usr/lib/gcc to see what your compiler probably thinks --build should be)
So, eg, if you're building for i686 on x86_64, do
./configure --host=i686-linux-gnu
(And use the -m32 options in CFLAGS, etc., but it sounds like you already have that ready.)
Whereas if you're building for x86_64 on i686, do
./configure --host=x86_64-linux-gnu
(You can build for all kinds of crazy hosts: rs6000-ibm-aix, sparc-sun-solaris, mips-idt-ecoff, etc..., assuming you have the appropriate gcc cross-compilers installed...)
GNU's page on it is here:
http://www.gnu.org/software/automake/manual/html_node/Cross_002dCompilation.html

What do I need to debug pthreads?

I want to debug pthreads on my custom linux distribution but I am missing something. My host is Ubuntu 12.04, my target is an i486 custom embedded Linux built with a crosstool-NG cross compiler toolset, the rest of the OS is made with Buildroot.
I will list the facts:
I can run multi-threaded applications on my target
Google Breakpad fails to create a crash report when I run a multi-threaded application on the target. The exact same application with the exact same build of Breakpad libraries will succeed when I run it on my host.
GDB fails to debug multithreaded applications on my target.
e.g.
$./gdb -n -ex "thread apply all backtrace" ./a.out --pid 716
dlopen failed on 'libthread_db.so.1' - /lib/libthread_db.so.1: undefined symbol: ps_lgetfpregs
GDB will not be able to debug pthreads.
GNU gdb 6.8
I don't think ps_lgetfpregs is a problem because of this.
My crosstool build created the libthread_db.so file, and I put it on the target.
My crosstool build created the gdb for my target, so it should have been linked against the same libraries that I run on the target.
If I run gdb on my host, against my test app, I get a backtrace of each running thread.
I suspect the problem with Breakpad is related to the problem with GDB, but I cannot substantiate this. The only commonality is lack of multithreaded debug.
There is some crucial difference between my host and target that stops me from being able to debug pthreads on the target.
Does anyone know what it is?
EDIT:
Denys Dmytriyenko from TI says:
Normally, GDB is not very picky and you can mix and match different
versions of gdb and gdbserver. But, unfortunately, if you need to
debug multi-threaded apps, there are some dependencies for specific
APIs...
For example, this is one of the messages you may see if you didn't
build GDB properly for the thread support:
dlopen failed on 'libthread_db.so.1' - /lib/libthread_db.so.1:
undefined symbol: ps_lgetfpregs GDB will not be able to debug
pthreads.
Note that this error is the same as the one that I get but he doesn't go in to detail about how to build GDB "properly".
and the GDB FAQ says:
(Q) GDB does not see any threads besides the one in which crash occurred;
or SIGTRAP kills my program when I set a breakpoint.
(A) This frequently
happen on Linux, especially on embedded targets. There are two common
causes:
you are using glibc, and you have stripped libpthread.so.0
mismatch between libpthread.so.0 and libthread_db.so.1
GDB itself does
not know how to decode "thread control blocks" maintained by glibc and
considered to be glibc private implementation detail. It uses
libthread_db.so.1 (part of glibc) to help it do so. Therefore,
libthread_db.so.1 and libpthread.so.0 must match in version and
compilation flags. In addition, libthread_db.so.1 requires certain
non-global symbols to be present in libpthread.so.0.
Solution: use
strip --strip-debug libpthread.so.0 instead of strip libpthread.so.0.
I tried a non-stripped libpthread.so.0 but it didn't make a difference. I will investigate any mismatch between pthread and thread_db.
This:
dlopen failed on 'libthread_db.so.1' - /lib/libthread_db.so.1: undefined symbol: ps_lgetfpregs
GDB will not be able to debug pthreads.
means that the libthread_db.so.1 library was not able to find the symbol ps_lgetfpregs in gdb.
Why?
Because I built gdb using Crosstoolg-NG with the "Build a static native gdb" option and this adds the -static option to gcc.
The native gdb is built with the -rdynamic option and this populates the .dynsym symbol table in the ELF file with all symbols, even unused ones. libread_db uses this symbol table to find ps_lgetfpregs from gdb.
But -static strips the .dynsym table from the ELF file.
At this point there are two options:
Don't build a static native gdb if you want to debug threads.
Build a static gdb and a static libthread_db (not tested)
Edit:
By the way, this does not explain why Breakpad in unable to debug multithreaded applications on my target.
Just a though... To use the gdb debugger, you need to compile your code with -g option. For instance, gcc -g -c *.c.

Debugging shared libraries with gdbserver

I am using gdbserver on target and CodeSourcery IDE. My hardware is a gumstix with a omap3530.
I can step through code in my main application but if I attempt to step into a function in a shared library I get memory address and a debugger terminates.
This is my library that is compiled and copied to the /lib folder on the target system.(it does have debug symbols) I have attempted to use the .gbdinit file to set solib-absolute-prefix /lib
Here are the warnings from the gdb trace:
903,056 13-gdb-set sysroot-on-target /lib
903,065 13^done
903,065 (gdb)
903,065 14-target-select remote 192.168.1.101:2345
903,114 =thread-group-started,id="i1",pid="42000"
903,114 =thread-created,id="1",group-id="i1"
903,115 15-list-thread-groups --available
903,120 16-list-thread-groups
903,128 &"warning: Unable to find dynamic linker breakpoint function.\nGDB will be unable to debug shared library initializers\nand track explicitly loaded dynamic code."
903,128 &"\n"
Which leads to
903,395 &"Error while mapping shared library sections:\n"
903,397 &"/lib/libCoreLib.so: Invalid argument.\n"
903,399 =library-loaded,id="/lib/libCoreLib.so",target-name="/lib/libCoreLib.so",hostname="/lib/libCoreLib.so",low-address="0x0",high-address="0x0",symbols-loaded="0",thread-group="i1"
You can debug with the library installed on your host, provided the debugging machine is also the development machine. In that case, you use set sysroot instead of set sysroot-on-target. For example :
set sysroot /home/username/.../rootfs/
where /home/username/.../rootfs/ contains a copy of your target filesystem
I think you should also specify / instead of /lib
Target with debug symbols
This is the simplest method to get working, and it is specially useful when you are developing one particular shared library.
First copy the test executable and shared library to the target with debug information:
check with readelf ----debug-dump=decodedline libmyib.so: How can I tell if a library was compiled with -g?
I recommend using an NFS server on host, so that the compiled output gets automatically uploaded after compilation
Then on target:
gdbserver --multi :1234 ./executable_name
Host:
arm-linux-gnueabihf-gdb -q -nh \
-ex "target extended-remote target-hostname-or-ip:1234" \
-ex "file ./executable_name" \
-ex 'tb main' \
-ex 'c' \
-ex 'set solib-search-path .'
sharedlibrary libmylib.so also works.
The problem I had was that gdbserver stops at the dynamic loader, before main, and the dynamic libraries are not yet loaded at that point, and so GDB does not know where the symbols will go in memory yet.
GDB appears to have some mechanisms to automatically load shared library symbols, and if I compile for host, and run gdbserver locally, running to main is not needed. But on the ARM target, that is the most reliable thing to do.
Target gdbserver 7.12-6, host arm-linux-gnueabihf-gdb 7.6.1 from Linaro.
Target libraries without debug symbols
It is common to strip target libraries before deployment on embedded targets, since debug information makes them way larger.
For example, Buildroot does that by default, but you can disable it with BR2_STRIP_none=y.
You can identify this scenario by running:
info shared
Which shows something like:
From To Syms Read Shared Object Library
0x00007ffff7df7f90 0x00007ffff7dfcdd7 Yes (*) target:/lib/ld64-uClibc.so.0
0x00007ffff7b3a9b0 0x00007ffff7bbe05d Yes (*) target:/lib/libc.so.0
(*): Shared library is missing debugging information.
so there are asterisks (*) for both of the libraries which says that debug information is missing.
If that is the case, then you have to tell GDB to use the shared libraries on host before they were stripped.
Buildroot for example makes that easy for us, as it maintains the staging directory which contains the shared libraries before they were stripped and in the same relative paths as in the target:
set sysroot buildroot/output/staging/
When this option is set, gdb immediately searches for libraries in the host instead of target, and finds /lib/libc.so.0 at the path buildroot/output/staging/ + /lib/libc.so.0:
Reading symbols from buildroot/output/staging/lib/ld64-uClibc.so.0...done.
Reading symbols from buildroot/output/staging/lib/libc.so.0...done.
TODO: I don't think you can set more than one sysroot, so all your shared libraries must be placed in their correct relative paths as in the target image.
If you check the bad default sysroot, you will see:
show sysroot
give:
target:
which means that gdb searches for shared libraries on the target root / by default.
Similar issue was encountered while debugging. The debug was hanging up. Configuration is as follows
Host: Ubuntu 12.04LTS
IDE: Eclipse Kepler
Target: Beaglebone Black / ARM A8
OS: Angstrom
Solution
Update libraries and includes
Select properties for project in Eclipse
C/C++ General > Paths and Symbols > (include TAB) GNU C > Add > Files
systems > / > usr Change from /usr/lib/gcc/i686-linux-gnu/4/6/include
to /usr/arm-linux-gnueabi/include
C/C++ General > Paths and Symbols > (Include TAB) GNU C++> Add >
Files systems > / > usr /usr/arm-linux-gnueabi/include/c++/4.6.3/arm-linux-gnueabi
C/C++ General > Paths and Symbols > (Library Paths TAB) > Add > Files
systems > / > usr /usr/arm-linux-gnueabi/lib
Good day,
If the 'debug-file-directory' variable in GDB is set incorrectly,
then the reported error messages contains:
warning: Unable to find dynamic linker breakpoint function.
The root filesystem of the target is located on my host PC at
/opt/arm-linux-gnueabihf-rootfs
The following two commands helped me to get remote debugging working
via gdbserver using GDB (v7.11.1):
set debug-file-directory /opt/arm-linux-gnueabihf-rootfs/usr/lib/debug
set sysroot /opt/arm-linux-gnueabihf-rootfs
I've noticed that if 'sysroot' has a trailing slash in the path,
then GDB fails to use it.You will see this (incorrect output) after connecting to the remote target:
Reading /lib/ld-linux-armhf.so.3 from remote target...
or
Reading symbols from /opt/arm-linux-gnueabihf-rootfs/lib/ld-linux-
armhf.so.3...(no debugging symbols found)...done
instead of the correct output:
Reading symbols from /opt/arm-linux-gnueabihf-rootfs/lib/ld-linux-
armhf.so.3...
Reading symbols from /opt/arm-linux-gnueabihf-rootfs/usr/lib/debug/
lib/arm-linux-gnueabihf/ld-2.23.so...done.
Regards,
Frikkie Thirion

Resources