In Linux, to get a backtrace you can use backtrace() library call, but it only returns backtrace of current thread. Is there any way to get a backtrace of some other thread, assuming I know it's TID (or pthread_t) and I can guarantee it sleeps?
It seems that libunwind (http://www.nongnu.org/libunwind/) project can help. The problem is that it is not supported by CentOS, so I prefer not to use it.
Any other ideas?
Thanks.
I implemented that myself here.
Initially, I wanted to implement something similar as suggested here, i.e. getting somehow the top frame pointer of the thread and unwinding it manually (the linked source is derived from Apples backtrace implementation, thus might be Apple-specific, but the idea is generic).
However, to have that safe (and the source above is not and may even be broken anyway), you must suspend the thread while you access its stack. I searched around for different ways to suspend a thread and found this, this and this. Basically, there is no really good way. The common hack, also used by the Hotspot JAVA VM, is to use signals and sending a custom signal to your thread via pthread_kill.
So, as I would need such signal-hack anyway, I can have it a bit simpler and just use backtrace inside the called signal handler which is executed in the target thread (as also suggested here by sandeep). This is basically what my implementation is doing.
If you are also interested in printing the backtrace, i.e. get some useful debugging information (function name, source code filename, source code line number, ...), read here about an extended backtrace_symbols based on libbfd. Or just see the source here.
Signal Handling with the help of backtrace can solve your purpose.
I mean if you have a PID of the Thread, you can raise a signal for that thread. and in the handler you can use the backtrace. since the handler would be executing in that partucular thread, the backtrace there would be the output what you are needed.
gdb provides these facilities for debugging multi-thread programs:
automatic notification of new threads
‘thread thread-id’, a command to switch among threads
‘info threads’, a command to inquire about existing threads
‘thread apply [thread-id-list] [all] args’, a command to apply a command to a list of threads
thread-specific breakpoints
‘set print thread-events’, which controls printing of messages on thread start and exit.
‘set libthread-db-search-path path’, which lets the user specify which libthread_db to use if the default choice isn't compatible with the program.
So just goto required thread in GDB by cmd: 'thread thread-id'.
Then do 'bt' in that thread context to print the thread backtrace.
Related
I have a code running in Python 3.7.4 which forks off multiple processes. I believe I'm hitting a known issue (issue6721: https://github.com/python/cpython/issues/50970). I setup the child process to send "progress report" through a pipe to the parent process and noticed that sometimes a log statement doesn't get printed and that the code gets stuck in a deadlock situation.
After reading issue6721, I'm not sure I'm still understanding why parent might hold logger Handler lock after a log statement is done execution (i.e the line that logs is executed and the execution has moved to the next line of code). I totally get it that in the context of C++, the compiler might re-arrange instructions. Not fully understand it in context of Python. In C++ I can have barrier instructions to stop the compiler moving instructions beyond a point. Is there something similar that can be done in Python to avoid having a lock getting copied to child process?
I have seen solutions using "atfork" which is a library that seems not supported (so I can't really use it).
Does anyone know a reliable and standard solution to this problem?
I have a tcl proc called run_expect that I use to run basic tcl expect flow: spawn <device>, send <cmd>, expect <string>. Now I need to run this code from 2 threads running in parallel, I did the following attemps:
when I tried to write the multi-threaded proc which simply calls run_expect I got the error of unknown command run_expect from the thread's context/scope.
I tried to take the implementation of the run_expect proc and put it in the thread itself, but then I encountered another issue that the thread doesn't seem to see expect library as the other procs and complains on: "invalid command name "spawn".
I tried then to do package require Expect from the thread itself, but got Segmentation fault: 11 error.
Tried to update the ::audio_path variable of the thread to be same as main context but also didn't help making the package require work (::thread::send -async [lindex $tids 0] [list set ::auto_path $::auto_path])
Is there anyway to call any already existing proc from a thread?
if not, is moving the code into the thread is the write solution? and how can I get the thread to know the packages / commands loaded?
Each thread in Tcl is almost totally isolated from all other threads. They don't share any commands (including procedures) or variables. The easiest way to manage things in multiple threads is to put the code to be in each thread into its own Tcl file and to tell the worker threads to source that as part of starting up.
However…
The Expect package is not thread safe; you've provided clear evidence of that. I don't know the details of why this is so. This means that if you want multithreaded expecting, your easiest approach is to use several processes instead. Tcl's good at managing subprocesses via asynchronous pipelines, and when everything is designed to work that way, you don't need to use Expect in the parent to manage things. You could also use the comm package to do the communications with the subprocesses.
This is what says on http://invisible-island.net/ncurses/ncurses.faq.html#multithread
If you have a program which uses curses in more than one thread, you will almost certainly see odd behavior. That is because curses relies upon static variables for both input and output. Using one thread for input and other(s) for output cannot solve the problem, nor can extra screen updates help. This FAQ is not a tutorial on threaded programming.
Specifically, it mentions it is not safe even if input and output are done on separate threads. Would it be safe if we further use a mutex for the whole ncurses library so that at most one thread can be calling any ncurses function at a time? If not, what would be other cheap workarounds to use ncurses safely in multi-thread application?
I'm asking this question because I notice a real application often has its own event loop but relies on ncurses getch function to get keyboard input. But if the main thread is block waiting in its own event loop, then it has no chance to call getch. A seemingly applicable solution is to call getch in a different thread, which hasn't caused me a problem yet, but as what says above is actually not safe, and was verified by another user here. So I'm wondering what is the best way to merge getch into an application's own event loop.
I'm considering making getch non-blocking and waking up the main thread regularly (every 10-100 ms) to check if there is something to read. But this adds an additional delay between key events and makes the application less responsive. Also, I'm not sure if that would cause any problems with some ncurses internal delay such as ESCDELAY.
Another solution I'm considering is to poll stdin directly. But I guess ncurses should also be doing something like that and reading the same stream from two different places looks bad.
The text also mentions the "ncursest" or "ncursestw" libraries, but they seem to be less available, for example, if you are using a different language binding of curses. It would be great if there is a viable solution with the standard ncurses library.
Without the thread-support, you're out of luck for using curses functions in more than one thread. That's because most of the curses calls use static or global data. The getch function for instance calls refresh which can update the whole screen—using the global pointers curscr and stdscr. The difference in the thread-support configuration is that global values are converted to functions and mutex's added.
If you want to read stdin from a different thread and run curses in one thread, you probably can make that work by checking the file descriptor (i.e., 0) for pending activity and alerting the thread which runs curses to tell it to read data.
My goal is to set virtual memory page permissions (as if the forked process called mprotect) from the parent process. Can this be done with ptrace(1) or by some other magic?
Thanks!
It can be done (via ptrace() indeed; gdb can do this), but not without a lot of finagling, since in order to call a function in another process, you basically have to setup its registers and stack, etc. for execution, and then continue the process, which will execute the function. One program I know off the top of my head that might have some useful source/methodology for you to look at is injectso. If you do look at injectso, look at the inject_code() functions.
In addition, calling conventions vary by platform, so you'd have to re-jigger your code for each architecture/OS, etc.
I'm debugging a reference leak in a GObject-based application. GObject has a simple built-in mechanism to help with such matters: you can set the g_trap_object_ref variable in gobject.c to the object that you care about, and then every ref or unref of that object will hit a breakpoint instruction (via G_BREAKPOINT()).
So sure enough, the program does get stopped, with gdb reporting:
Program received signal SIGTRAP, Trace/breakpoint trap.
g_object_ref (_object=0x65f090) at gobject.c:2606
2606 old_val = g_atomic_int_exchange_and_add ((int *)&object->ref_count, 1);
(gdb) _
which is a great start. Now, normally I'd script some commands to be run at a breakpoint I manually set using commands 3 (for breakpoint 3, say). But the equivalent for SIGTRAP, namely handle SIGTRAP, doesn't give me the option of doing anything particularly interesting. Is there a good way to do this?
(I'm aware that there are other ways to debug reference leaks, such as setting watchpoints on the object's ref_count field, refdbg, scripting regular breakpoints on g_object_ref() and g_object_unref(). I'm about to go try of those now. I'm looking specifically for a way to script a response to SIGTRAP. It might come in useful in other situations, too, and I'd be surprised if gdb doesn't support this.)
Do you want to show some values and continue execution of the program? In that case, just define a macro that displays the values you're interested in, continues execution and calls itself recursively:
define c
echo do stuff\n
continue
c
end
GDB doesn't support it.
In general, attaching a command script to signal makes little sense -- your program could be receiving SIGTRAP in any number of places, and the command will not know whether a particular SIGTRAP came in in expected context or not.