what's the detail about gdb, does it hold one thread? - multithreading

What's the detail about gdb, does it hold one thread when debugging code?
I set an exit flag in the main thread, and I have joined other threads before printing the flag. When I run the debug version using gdb after compiling with argument "-g", it looks:
[Thread debugging using libthread_db enabled]
Input number of threads:5
Main thread id: 0xb7fda6c0,
[New Thread 0xb7fd9b70 (LWP 9276)]
[New Thread 0xb75d8b70 (LWP 9277)]
[New Thread 0xb6bd7b70 (LWP 9278)]
[New Thread 0xb61d6b70 (LWP 9279)]
[New Thread 0xb57d5b70 (LWP 9280)]
I am thread 0xb6bd7b70, myorder 2, thread_exit.
I am thread 0xb61d6b70, myorder 3, thread_exit.
I am thread 0xb7fd9b70, myorder 0, thread_exit.
I am thread 0xb57d5b70, myorder 4, thread_exit.
I am thread 0xb75d8b70, myorder 1, thread_exit.
[Thread 0xb61d6b70 (LWP 9279) exited]
[Thread 0xb6bd7b70 (LWP 9278) exited]
[Thread 0xb75d8b70 (LWP 9277) exited]
[Thread 0xb7fd9b70 (LWP 9276) exited]
Main: completed join with thread 0xb7fd9b70 having a status of 0
Main: completed join with thread 0xb75d8b70 having a status of 1
Main: completed join with thread 0xb6bd7b70 having a status of 2
Main: completed join with thread 0xb61d6b70 having a status of 3
Main: completed join with thread 0xb57d5b70 having a status of 4
Main: lock will be destroy
Main: 9273, tlist will be free
Main exit.
[Thread 0xb57d5b70 (LWP 9280) exited]
Program exited normally.
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.7.el6.i686 libgcc-4.4.4-13.el6.i686
GDB gives information about the "create" and "exit" of threads. But there is always one thread exited after calling pthread_join() and printing "Main exit" in main thread. Why? Is the thread which gdb work with?
Run the release edition, nothing special:
Input number of threads:5
Main thread id: 0xb77176c0,
I am thread 0xb5913b70, myorder 3, thread_exit.
I am thread 0xb4f12b70, myorder 4, thread_exit.
I am thread 0xb6314b70, myorder 2, thread_exit.
I am thread 0xb6d15b70, myorder 1, thread_exit.
I am thread 0xb7716b70, myorder 0, thread_exit.
Main: completed join with thread 0xb7716b70 having a status of 0
Main: completed join with thread 0xb6d15b70 having a status of 1
Main: completed join with thread 0xb6314b70 having a status of 2
Main: completed join with thread 0xb5913b70 having a status of 3
Main: completed join with thread 0xb4f12b70 having a status of 4
Main: lock will be destroy
Main: tdata list will be free
Main exit.

gdb, does it hold one thread when debug code?
No.
there is always one thread exited after calling pthread_join() and printing "Main exit" in main thread. Why?
The notification that glibc sends to GDB to tell it that a thread has exited is sent as (nearly) the last thing a thread does. In particular, that notification is sent after the exiting thread has notified the main thread that it has exited (i.e. after the exiting thread wakes the main thread). Therefore, it's not at all surprising that main thread wakes up and prints Main exit before GDB notices that the exiting thread has in fact exited.
I have run it many times, there is only one thread after "Main exit" for each time.
That proves nothing. If you run your program 1,000,000 times, there will probably be at least one run where GDB will notice the thread exiting before Main exit is printed. The ordering is likely dependent on how many processors your system has, and how busy they are. There is really nothing (interesting) to see here.

Related

is there any way to get the start time of the thread in gdb core dump

i am trying to find oldest thread in Linux core dump. So i need get start time of each thread that are running. So is there any way to get the start time of thread.
i tried to some command in gdb core file but that gives each call stack of threads but not start time.
Regards
Rabi
i am trying to find oldest thread in Linux core dump. So i need get start time of each thread that are running.
Your conclusion does not follow from your previous statement.
So is there any way to get the start time of thread.
No: that time is not recorded anywhere in the thread structure.
However, in general Linux allocates thread ids in increasing order, so sorting by thread id should give you the order you desire. For example:
(gdb) info thread
Id Target Id Frame
* 1 Thread 0x7fa91e769740 (LWP 2128629) __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
2 Thread 0x7fa91d766700 (LWP 2128632) 0x0000557aa5787165 in fn ()
3 Thread 0x7fa91af61700 (LWP 2128637) 0x0000557aa5787165 in fn ()
4 Thread 0x7fa91df67700 (LWP 2128631) 0x0000557aa5787165 in fn ()
5 Thread 0x7fa91cf65700 (LWP 2128633) 0x0000557aa5787165 in fn ()
6 Thread 0x7fa91bf63700 (LWP 2128635) 0x0000557aa5787165 in fn ()
7 Thread 0x7fa91c764700 (LWP 2128634) 0x0000557aa5787165 in fn ()
8 Thread 0x7fa91a760700 (LWP 2128638) 0x0000557aa5787165 in fn ()
9 Thread 0x7fa91b762700 (LWP 2128636) 0x0000557aa5787165 in fn ()
10 Thread 0x7fa919f5f700 (LWP 2128639) 0x0000557aa5787165 in fn ()
Here you can see that thread 1 was created first, followed by thread 4, 2, 5, 7, etc (LWP is the Linux thread id here).

OpenCV app compiled with C++11 creates extra thread

I'm debugging a OpenCV app compiled with C++11 (I use OpenCV 2.4.10). The app has two threads that do some image processing on the CPU (no GPU functions used but I also included libopencv_gpu.so in the linked libraries).
Using gdb I noticed that instead of just two threads (the main process thread and another thread created by the main process thread) I found 3 threads running:
(gdb) info threads
Id Target Id Frame
78 Thread 0x7fffe2ff5700 (LWP 20531) "app_name" 0x00007ffff5bb2f3d in nanosleep () at ../sysdeps/unix/syscall-template.S:81
2 Thread 0x7fffe3c42700 (LWP 20454) "app_name" 0x00007ffff5bdf12d in poll () at ../sysdeps/unix/syscall-template.S:81
* 1 Thread 0x7ffff7fab800 (LWP 20450) "app_name" 0x00007ffff5bb2f3d in nanosleep () at ../sysdeps/unix/syscall-template.S:81
Thread 1 and 78 (using gdb ID) are executing my code. I added a sleep call in each one so I can make sure that those are my threads.
Thread 2 (using gdb ID) is created before entering the main function of the main process I believe. As far as I could debug this, thread with ID 2 just calls poll() function all the time.
I'm new to gdb and maybe you can tell me how to find out who creates this thread and what is it's purpose? Is this OpenCV related or C++11? When I compile the same app using Opencv4Tegra and run it on a Tegra K1 board, thread number 2 does not exist.
EDIT
This is the backtrace when creating thread number 2. It seems that libusb creates this but I don't know why yet:
(gdb) backtrace
#0 __pthread_create_2_1 (newthread=0x7fffea79c438, attr=0x0, start_routine=0x7fffea5941c0, arg=0x0) at pthread_create.c:466
#1 0x00007fffea5943df in ?? () from /lib/x86_64-linux-gnu/libusb-1.0.so.0
#2 0x00007fffea5926a5 in ?? () from /lib/x86_64-linux-gnu/libusb-1.0.so.0
#3 0x00007fffea58b715 in libusb_init () from /lib/x86_64-linux-gnu/libusb-1.0.so.0
#4 0x00007ffff2f06a0e in ?? () from /usr/lib/x86_64-linux-gnu/libdc1394.so.22
#5 0x00007ffff2ef5465 in dc1394_new () from /usr/lib/x86_64-linux-gnu/libdc1394.so.22
#6 0x00007ffff6f615e9 in CvDC1394::CvDC1394() () from /usr/local/lib/libopencv_highgui.so.2.4
#7 0x00007ffff6f373f0 in _GLOBAL__sub_I_cap_dc1394_v2.cpp () from /usr/local/lib/libopencv_highgui.so.2.4
#8 0x00007ffff7dea13a in call_init (l=<optimized out>, argc=argc#entry=3, argv=argv#entry=0x7fffffffdcd8, env=env#entry=0x7fffffffdcf8) at dl-init.c:78
#9 0x00007ffff7dea223 in call_init (env=<optimized out>, argv=<optimized out>, argc=<optimized out>, l=<optimized out>) at dl-init.c:36
#10 _dl_init (main_map=0x7ffff7ffe1c8, argc=3, argv=0x7fffffffdcd8, env=0x7fffffffdcf8) at dl-init.c:126
#11 0x00007ffff7ddb30a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
(gdb) quit

How to change executing thread in gdb?

I'm currently debugging kernel code using KGDB.
Whenever I break in I naturally jump to the interrupt handler for kgdb.
Under GDB I ran the following command.
info threads
and the output would be
7 Thread 7 (rcu_sched) 0x0000000000000000 in irq_stack_union ()
6 Thread 5 (kworker/0:0H) 0x0000000000000000 in irq_stack_union ()
5 Thread 3 (ksoftirqd/0) 0x0000000000000000 in irq_stack_union ()
4 Thread 2 (kthreadd) 0x0000000000000000 in irq_stack_union ()
3 Thread 1 (init) 0x0000000000000000 in irq_stack_union ()
2 Thread 3754 (Xorg) 0x0000000000000000 in irq_stack_union ()
* 1 Thread 4294967294 (shadowCPU0) kgdb_breakpoint ()
at kernel/debug/debug_core.c:1042
I would then jump through the code expecting to end up in a different thread (I'm interested in Xorg) however after i step through the code the next executing thread becomes cpu idle.
info thread
* 1 Thread 4294967294 (shadowCPU0) cpu_idle_loop () at kernel/cpu/idle.c:116
How can I switch my debug context to Xorg or any other thread, additionally what does irq_stack_union () mean. Thread is idle pending interrupts?
According to the offical documentation is is just thread threadno
https://sourceware.org/gdb/onlinedocs/gdb/Threads.html

Printing thread hierarchy with GDB

I have a multithreaded program that I'm trying to debug. When I run info thread in GDB, I get the following:
(gdb) info thread
Id Target Id Frame
8 Thread 0x7fffe77fd700 (LWP 17425) "SocketWriter" 0x00007ffff7bc9b2f in pthread_cond_wait##GLIBC_2.3.2 () from /usr/lib/libpthread.so.0
7 Thread 0x7fffe73fc700 (LWP 17426) "SocketWriter" 0x00007ffff7bc9b2f in pthread_cond_wait##GLIBC_2.3.2 () from /usr/lib/libpthread.so.0
6 Thread 0x7fffe7fff700 (LWP 17423) "SocketReader" 0x00007ffff7bcc66d in read () from /usr/lib/libpthread.so.0
5 Thread 0x7fffe7bfe700 (LWP 17424) "SocketReader" 0x00007ffff7bcc66d in read () from /usr/lib/libpthread.so.0
* 4 Thread 0x7ffff4810700 (LWP 17422) "unittest" 0x00007ffff7bcc38c in __lll_lock_wait () from /usr/lib/libpthread.so.0
3 Thread 0x7ffff4c11700 (LWP 17421) "receiver" 0x00007ffff7bcc38c in __lll_lock_wait () from /usr/lib/libpthread.so.0
2 Thread 0x7ffff5a3b700 (LWP 17420) "unittest" 0x00007ffff634e553 in select () from /usr/lib/libc.so.6
1 Thread 0x7ffff7fc9780 (LWP 17419) "unittest" 0x00007ffff7bc9b2f in pthread_cond_wait##GLIBC_2.3.2 () from /usr/lib/libpthread.so.0
It would be excellent if I could make GDB display the parent/child relationships between the threads, something like the following:
(gdb) info thread
Id Target Id Frame
1 Thread 0x7ffff7fc9780 (LWP 17419) "unittest" 0x00007ffff7bc9b2f in pthread_cond_wait##GLIBC_2.3.2 () from /usr/lib/libpthread.so.0
3 Thread 0x7ffff4c11700 (LWP 17421) "receiver" 0x00007ffff7bcc38c in __lll_lock_wait () from /usr/lib/libpthread.so.0
8 Thread 0x7fffe77fd700 (LWP 17425) "SocketWriter" 0x00007ffff7bc9b2f in pthread_cond_wait##GLIBC_2.3.2 () from /usr/lib/libpthread.so.0
6 Thread 0x7fffe7fff700 (LWP 17423) "SocketReader" 0x00007ffff7bcc66d in read () from /usr/lib/libpthread.so.0
2 Thread 0x7ffff5a3b700 (LWP 17420) "unittest" 0x00007ffff634e553 in select () from /usr/lib/libc.so.6
5 Thread 0x7fffe7bfe700 (LWP 17424) "SocketReader" 0x00007ffff7bcc66d in read () from /usr/lib/libpthread.so.0
* 4 Thread 0x7ffff4810700 (LWP 17422) "unittest" 0x00007ffff7bcc38c in __lll_lock_wait () from /usr/lib/libpthread.so.0
7 Thread 0x7fffe73fc700 (LWP 17426) "SocketWriter" 0x00007ffff7bc9b2f in pthread_cond_wait##GLIBC_2.3.2 () from /usr/lib/libpthread.so.0
For example, thread 3 is the parent of threads 8, 6, and 2, and thread 1 is the parent of everything.
Does such functionality exist? I have not seen reference to it, if it does.
gdb doesn't print this information because it doesn't exist in your program -- there is no way for gdb to discover it once the threads have been created.
There are maybe two ways it could be done.
First, you could set a breakpoint on the thread-creation function and record the information. This is readily done from Python. Then you can write a new command, also in Python, to format the output the way you like.
The problem with this approach is that it won't work if you "attach" to a running program. It will be too late to capture the information.
Another method is if you have extra information available in your program that describes the hierarchy. Then you can write a new command in Python that extracts this information to display things as you like.

how to block program executing when a new thread is created? ( with GDB)

EX: Thread A creates Thread B.
i wish to block thread A when it starts to create Thread B.
but i do not know how to do it with GDB.
Any help would be appreciated!
Thanks
Use the GDB non-stop mode and put a breakpoint at the line after your program calls pthread_create().
Reading symbols from bad_thread...done.
(gdb) set target-async 1
(gdb) set pagination off
(gdb) set non-stop on
(gdb) b pthread_create
Breakpoint 1 at 0x400570
(gdb) r
Starting program: bad_thread
[Thread debugging using libthread_db enabled]
Breakpoint 1, 0x0000000000400570 in pthread_create#plt ()
(gdb) bt
#0 0x0000000000400570 in pthread_create#plt ()
#1 0x00000000004006d8 in main () at bad_thread.c:21
(gdb) list
12 pthread_mutex_unlock(&x_mutex);
13
14 pthread_exit(NULL);
15 }
16
17 int
18 main () {
19 pthread_t tid1, tid2;
20
21 pthread_create(&tid1, NULL, add_thread, NULL);
(gdb) b bad_thread.c:22
Breakpoint 2 at 0x4006d8: file bad_thread.c, line 22.
(gdb) b add_thread
Breakpoint 3 at 0x400694: file bad_thread.c, line 10.
(gdb) c
Continuing.
[New Thread 0x40a00960 (LWP 26829)]
Breakpoint 2, main () at bad_thread.c:22
22 pthread_create(&tid2, NULL, add_thread, NULL);
(gdb)
Breakpoint 3, add_thread (arg=0x0) at bad_thread.c:10
10 pthread_mutex_lock(&x_mutex);
bt
#0 main () at bad_thread.c:22
(gdb)
To explain the above:
Turn on non-stop mode as per GDB docs
put a breakpoint at pthread_create() to determine where the program calls this
run and hit the breakpoint
backtrace and list to find the next line of code
put a breakpoint at the next line of code and at the thread start function (add_thread, in this case)
run
Notice we hit the breakpoint at the next line of code in the original thread. The new thread continues in the background and then hits the breakpoint at the thread start function.

Resources