I get the following message in gdb (version 7.1):
[Thread debugging using libthread_db enabled]
and the command reverse-step
results with the following error message:
(gdb) reverse-step
Target multi-thread does not support this command
I am debugging a serial code right now, so I definitely do not need multi-threading. Can I turn this off somehow so that I get the latest reverse-debug commands to work? Also, if the code is parallelised with OpenMPI, there will be no need for multi-thread debugging at all, right?
Edit: Is this set as a compilation flag that can be just excluded?
You don't mention which version of GDB you're using, but since a little while, the parameter libthread-db-search-path is available.
(gdb) set libthread-db-search-path /tmp
(gdb) start
Temporary breakpoint 1 at 0x400632: file threads.c, line 14.
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
will tell GDB to lookup it's helper library (libthread-db.so) in a directory where it isn't, so multithread debugging won't be enabled!
I'm not sure about OpenMPI parallel applications are multiprocesses (in contrast with OpenMP where they are multithreaded), so it won't change anything for you.
EDIT: Multithread debugging is usually only enabled when libpthread.so or equivalent is loaded by your process (ldd your-process to check if it's linked as a shared library) so if you don't need it, there might be a problem in your compilation script.
Related
When I attach to process with GDB I saw that warning:
Reading symbols from /CloneFileSystem/lib/ld-uClibc.so.0...
(No debugging symbols found in /CloneFileSystem/lib/ld-uClibc.so.0)
warning: Unable to find dynamic linker breakpoint function.
GDB will be unable to debug shared library initializers
and track explicitly loaded dynamic code.
Even I use set sysroot /CloneFileSystem with all the libraries there, that not help.
My process call to function in shared library and that function call to another function in another shared library.
For example ProcessFunc-> Lib1Func -> Lib2Func
I want to debug Lib2Func but when I run command
b Lib2Func
gdb can't set a breakpoint there event that library is already loaded
file myexe -> interpreter /lib/ld-uClibc.so.0
When I use info sharedlibrary in GDB I can't the value in From and To (address of loaded are empty)
You might want to try recompiling the library with a -g option. It's saying it can't find the debug info. -g provides for debugging info if you are using gcc. This is the first thing i'd check. If the library is big this may take awhile.
I am using a customized OS based on Oracle Linux 6.10. I have few customized command that helps to find information like firewall rules,hardware information but if I run those command then coredumps are generated.
There are 4 core files found on the one of my OracleLinux6.10 server like core.top.myrole.34089.bt
Below are the error that say it is generated by top.myrole 'Program terminated with signal 6, Aborted/No symbol table info available':
10/15/2019 13:54:57 core.top.myrole.34089.bt
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
top.myrole: No such file or directory.
Missing separate debuginfo for the main executable file
Try: yum --enablerepo='*-debug*' install
/usr/lib/debug/.build-id/81/7805fb3de3731d5c36c8f7da38040ee6d510ea
[New Thread 34090]
[New Thread 34089]
Core was generated by \top.myrole'.`
Program terminated with signal 6, Aborted.
#0 0x000000347960f00d in ?? ()
Thread 2 (Thread 34089):
#0 0x000000347960f00d in ?? ()
No symbol table info available.
#1 0x0000000000000001 in ?? ()
No symbol table info available.
After the above line 'No symbol table' error would now repeat for multiple threads.
Then found other coredump also generated while running a customized command 'hardwareshow'(this command is not linux command but it is customized within our distribution). The log shows that core was generated by '/usr/bin/perl /usr/product/bin/iprule show' :
10/14/2019 20:44:25, core.hardwareshow.8944.bt
warning: core file may not match specified executable file.
[New Thread 55835]
[Thread debugging using libthread_db enabled]
Core was generated by \/usr/bin/perl /usr/product/bin/iprule show--type=rule --protocol=ipv6 -'.Program terminated with signal 11, Segmentation fault.#0 0x0000003479a80829 in ?? () from /usr/lib64/perl5/CORE/libperl.so`
------------
10/14/2019 04:19:59, core.iprule.55835.bt
I checked the Perl version used in the machine and it seems it is Perl5 , (still needs to confirm if it`s 5.10).How do I address this issue, I heard from some Perl experts that Perl upgrade to a new version can help here still I would want to hear from multiple Linux developers/Perl Developers about this error.
The perl interpreter will coredump if: there is a bug in its implementation; if certain pathological Perl code patterns are used (e.g. infinite recursion calling tie or overload handlers); or if there is a bug in an external module that uses C-level code (like XS modules). Determining what the fault is can be hard. If you just want the problem to go away, then upgrading everything to a more recent version is probably the path of least resistance. Note that the perl 5.10.0 release is about 12 years old.
i had a problem in my native android application saying libc: Fatal signal 7 (SIGBUS) at 0x66f0001d (code=1), thread 30165 (xample.fft_test)
which i think is a memory problem on the device(nexus 4).
So, i tried to debug the application to know the source of this problem.
In eclipse, i get warning: Could not load shared library symbols for 94 libraries, e.g. /system/bin/linker.
Use the "info sharedlibrary" command to see the complete listing.
Do you need "set solib-search-path" or "set sysroot"?
warning: Unable to find dynamic linker breakpoint function.
GDB will retry eventurally. Meanwhile, it is likely
that GDB is unable to debug shared library initializers
or resolve pending breakpoints after dlopen().
what does that mean?
here's also the output of the allocation tracker in eclipse:
If your application.mk has flags:
APP_STL := gnustl_static
APP_OPTIM := debug
You have a problem with the compilation´s flags.
The warning means those libraries have not compiled with debug flags, and then, you have not debug info so you cannot debug. Maybe you can stop in breakpoints and follow step by step, but you cannot read in real time the value of the variables.
So, to solve the problem, you must re-compile the libraries that you want to read variables with debug information flags. In Cmake I usually use flag -d or -gdwarf-4 to crosscompile in Android.
I had the same issue and I solved it by using another phone.
I tested on a lot of phones. Finally, I find out that debug NDK on Nexus 6 is ok.
So I think maybe the issue is that the support for debugging NDK was removed on some phones.
I want to debug pthreads on my custom linux distribution but I am missing something. My host is Ubuntu 12.04, my target is an i486 custom embedded Linux built with a crosstool-NG cross compiler toolset, the rest of the OS is made with Buildroot.
I will list the facts:
I can run multi-threaded applications on my target
Google Breakpad fails to create a crash report when I run a multi-threaded application on the target. The exact same application with the exact same build of Breakpad libraries will succeed when I run it on my host.
GDB fails to debug multithreaded applications on my target.
e.g.
$./gdb -n -ex "thread apply all backtrace" ./a.out --pid 716
dlopen failed on 'libthread_db.so.1' - /lib/libthread_db.so.1: undefined symbol: ps_lgetfpregs
GDB will not be able to debug pthreads.
GNU gdb 6.8
I don't think ps_lgetfpregs is a problem because of this.
My crosstool build created the libthread_db.so file, and I put it on the target.
My crosstool build created the gdb for my target, so it should have been linked against the same libraries that I run on the target.
If I run gdb on my host, against my test app, I get a backtrace of each running thread.
I suspect the problem with Breakpad is related to the problem with GDB, but I cannot substantiate this. The only commonality is lack of multithreaded debug.
There is some crucial difference between my host and target that stops me from being able to debug pthreads on the target.
Does anyone know what it is?
EDIT:
Denys Dmytriyenko from TI says:
Normally, GDB is not very picky and you can mix and match different
versions of gdb and gdbserver. But, unfortunately, if you need to
debug multi-threaded apps, there are some dependencies for specific
APIs...
For example, this is one of the messages you may see if you didn't
build GDB properly for the thread support:
dlopen failed on 'libthread_db.so.1' - /lib/libthread_db.so.1:
undefined symbol: ps_lgetfpregs GDB will not be able to debug
pthreads.
Note that this error is the same as the one that I get but he doesn't go in to detail about how to build GDB "properly".
and the GDB FAQ says:
(Q) GDB does not see any threads besides the one in which crash occurred;
or SIGTRAP kills my program when I set a breakpoint.
(A) This frequently
happen on Linux, especially on embedded targets. There are two common
causes:
you are using glibc, and you have stripped libpthread.so.0
mismatch between libpthread.so.0 and libthread_db.so.1
GDB itself does
not know how to decode "thread control blocks" maintained by glibc and
considered to be glibc private implementation detail. It uses
libthread_db.so.1 (part of glibc) to help it do so. Therefore,
libthread_db.so.1 and libpthread.so.0 must match in version and
compilation flags. In addition, libthread_db.so.1 requires certain
non-global symbols to be present in libpthread.so.0.
Solution: use
strip --strip-debug libpthread.so.0 instead of strip libpthread.so.0.
I tried a non-stripped libpthread.so.0 but it didn't make a difference. I will investigate any mismatch between pthread and thread_db.
This:
dlopen failed on 'libthread_db.so.1' - /lib/libthread_db.so.1: undefined symbol: ps_lgetfpregs
GDB will not be able to debug pthreads.
means that the libthread_db.so.1 library was not able to find the symbol ps_lgetfpregs in gdb.
Why?
Because I built gdb using Crosstoolg-NG with the "Build a static native gdb" option and this adds the -static option to gcc.
The native gdb is built with the -rdynamic option and this populates the .dynsym symbol table in the ELF file with all symbols, even unused ones. libread_db uses this symbol table to find ps_lgetfpregs from gdb.
But -static strips the .dynsym table from the ELF file.
At this point there are two options:
Don't build a static native gdb if you want to debug threads.
Build a static gdb and a static libthread_db (not tested)
Edit:
By the way, this does not explain why Breakpad in unable to debug multithreaded applications on my target.
Just a though... To use the gdb debugger, you need to compile your code with -g option. For instance, gcc -g -c *.c.
I' ve a cross-gdb configured with --host=i686-pc-linux-gnu --target=powerpc-e300c3-linux-gnu. i can correctly debug an application on a remote board with gdbserver but i get an error about a version mismatch between libthread_db and libpthread so i can't debug threads correctly (gdb recognizes only one thread instead of three threads). Maybe it's is due to a different version of libc: on host machine i've libc2.15 and on target machine lib2.5. I tried to rebuid libc2.5 for host in order to link gdb against to it but it's an hell. Before i get crazy to rebuild it, could someone confirm that it's a libc problem?
I tried to rebuid libc2.5 for host in order to link gdb against to it
That's not what you need.
What you need, is for gdb to find and load libthread_db.so.1, that matches your target libpthread.so.0.
For this, you need to
build libc-2.5 for host, and
set GDB's libthread-db-search-path such that it finds the libthread_db.so.1 built in step 1.
You don't actually need to build the entire libc in step 1. Something like this should suffice:
mkdir build && cd build
../configure --prefix=/usr
make -C ../nptl_db objdir=`pwd`
Update:
i have GDB 6.6 and there isn't libthread-db-search-path. What is another way to specify that path?
That GDB will just dlopen("libthread_db.so.1", ...). So to make it find the right libthread_db.so.1, you need to adjust LD_LIBRARY_PATH. Using bash:
LD_LIBRARY_PATH=/tmp/glibc-2.5/build/nptl_db gdb /path/to/target/a.out