I found a trick to automate gdb commands "less disturbing", to do it I simply switch to the last thread in non-stop mode and stop that thread, then execute the requested commands and continue that thread and then return back to main thread.
But this trick won't work if the inferior only has only one thread. So I need to create a thread that runs forever at the background, by this way I'll be able to attach to it anytime I want without having to disturb the inferior.
The only solution that came to my mind was this:
1-)Break at syscall
2-)Allocate some memory with gdb to inject codes
3-)Replace syscall with the jmp instruction that points to the allocated memory
4-)pushad and execute the code that "somehow" creates the thread
5-)Replace the jmp with syscall back
6-)popad and jmp back to where syscall located
But this is way too hacky and I still have no idea about the "somehow" part. Is there a more elegant way to do this? Maybe gdb has some tools for it and I'm missing it. If not, how can I do the "somehow" part?
Yes, it is. I found an elegant way to do it, you don't have to do such hacky stuff. You can inject a thread like this:
1-)Write a code that crates a thread in any compilable language
2-)Compile it so a .so file(-g option should be passed, gdb will need debug symbols)
3-)Load it to the inferior with linux system call dlopen: call dlopen(".so path",int)
4-)Call any function you implemented in the .so file you injected by executing call funcname(), gdb will pick it up automatically when you press Tab(that's why you need debug symbols)
Related
In gdb, when debugging inside a function, we can use "finish" command to run to the end of a function.
My question is: how does gdb know the ending position of a function, especially when there's no debugging symbol to match source code "{}"?
I guess gdb looks for either "leave" or "mov %rbp, %rsp,pop %rbp" under x86 in order to judge whether it has reached the end of a function.
But the problem is,
(1) There're still some extra registers that needs to push/pop at the begin/end of a function call, depending on source code and ABI structure.
(2)The number of registers needs to be push/pop is decided during compilation phase, and I'm afraid this "number" information is not available throw binary executable file.
So, how does gdb determine, where is the end of a function call, so that "finish" command can jump to it?
Thanks!
gdb doesn't try to analyze the machine code. Instead, it unwinds the stack, finds the caller's PC, and sets a temporary breakpoint there. Then it lets the inferior run until the breakpoint is hit.
Due to the way gdb's unwinder is designed, this automatically handles finish from an inlined function as well (though there are still a few special cases in the code due to this).
Basically I am using ptrace to inject a shell code to a remote process for execution. But I found some weird behavior regarding RIP register.
What I do is I copy my shell code to the start address of where the program is mapped. Then I set the RIP using ptrace to the address where the start address is. And then I resume the target process for executing the code. Once the shell code finishes (by running int3) I will get signal and recover the code that I just modified.
It works fine except when the remote process is blocked inside of a system call like sleep. If the remote process is blocked inside of a system call at the moment I attach the process, after I set the RIP to where I want to execute my shell code and then resume the target process, I will observe that the RIP is actually 2 less than what the address that I put in the ptrace call. For example if I set the RIP to be 0x4000, once I resume it the RIP becomes 0x3ffe. Typically it crashes for my case due to the segment fault, obviously. But if I grab the register right after I set it without resuming the process, the RIP is the value that I just set. Currently I work around it by insert 2 nop instructions ahead of my shell code and always add 2 when I set the RIP. I just want to know is there anything that I miss for setting the RIP or my whole method for injecting code is totally unstable?
My dev box is Ubuntu14.04, kernel is 3.13.0-45-generic.
If I recall correctly, if you interrupt the process while it's blocked in a syscall, the program counter value, upon continuing, will be subtracted by sizeof(syscall instruction) by the kernel. So once you do a PTRACE_DETACH, the process will re-do the syscall it was interrupted from.
I overcome the problem the same way you did (always adding a tiny nop-sled and incrementing RIP).
I am interested in replacing a system call with a custom that I will implement in linux kernel 3.
I read that the sys call table is no longer exposed.
Any ideas?
any reference to this http://www.linuxtopia.org/online_books/linux_kernel/linux_kernel_module_programming_2.6/x978.html example but for kernel 3 will be appreciated :)
Thank you!
I would recommend using kprobes for this kind of job, you can easily break on any kernel address (or symbol...) and alter the execution path, all of this at runtime, with a kernel module if you need to :)
Kprobes work by dynamically replacing an instruction (e.g. first instruction of your syscall entry) by a break (e.g. int3 on x86). Inside the do_int3 handler, a notifier notifies kprobes, which in turn passes the execution to your registered function, from which point you can do almost anything.
A very good documentation is given in Documentation/kprobes.txt so as a tiny example in samples/kprobes/kprobes_example.c (in this example they break on do_fork to log each fork on the system). It has a very simple API and is very portable nowdays.
Warning: If you need to alter the execution path, make sure your kprobes are not optimized (i.e. a jmp instruction to your handler replaces the instruction you break onto instead of an int3) otherwize you won't be able to really alter the execution easily (after the ret of your function, the syscall function will still be executed as usual). If you are only interested in tracing, then this is fine and you can safely ignore this issue.
Write a LKM that would be better optio.What do you mean by replace,do you want to add a new one.
In Linux, to get a backtrace you can use backtrace() library call, but it only returns backtrace of current thread. Is there any way to get a backtrace of some other thread, assuming I know it's TID (or pthread_t) and I can guarantee it sleeps?
It seems that libunwind (http://www.nongnu.org/libunwind/) project can help. The problem is that it is not supported by CentOS, so I prefer not to use it.
Any other ideas?
Thanks.
I implemented that myself here.
Initially, I wanted to implement something similar as suggested here, i.e. getting somehow the top frame pointer of the thread and unwinding it manually (the linked source is derived from Apples backtrace implementation, thus might be Apple-specific, but the idea is generic).
However, to have that safe (and the source above is not and may even be broken anyway), you must suspend the thread while you access its stack. I searched around for different ways to suspend a thread and found this, this and this. Basically, there is no really good way. The common hack, also used by the Hotspot JAVA VM, is to use signals and sending a custom signal to your thread via pthread_kill.
So, as I would need such signal-hack anyway, I can have it a bit simpler and just use backtrace inside the called signal handler which is executed in the target thread (as also suggested here by sandeep). This is basically what my implementation is doing.
If you are also interested in printing the backtrace, i.e. get some useful debugging information (function name, source code filename, source code line number, ...), read here about an extended backtrace_symbols based on libbfd. Or just see the source here.
Signal Handling with the help of backtrace can solve your purpose.
I mean if you have a PID of the Thread, you can raise a signal for that thread. and in the handler you can use the backtrace. since the handler would be executing in that partucular thread, the backtrace there would be the output what you are needed.
gdb provides these facilities for debugging multi-thread programs:
automatic notification of new threads
‘thread thread-id’, a command to switch among threads
‘info threads’, a command to inquire about existing threads
‘thread apply [thread-id-list] [all] args’, a command to apply a command to a list of threads
thread-specific breakpoints
‘set print thread-events’, which controls printing of messages on thread start and exit.
‘set libthread-db-search-path path’, which lets the user specify which libthread_db to use if the default choice isn't compatible with the program.
So just goto required thread in GDB by cmd: 'thread thread-id'.
Then do 'bt' in that thread context to print the thread backtrace.
I'm debugging a reference leak in a GObject-based application. GObject has a simple built-in mechanism to help with such matters: you can set the g_trap_object_ref variable in gobject.c to the object that you care about, and then every ref or unref of that object will hit a breakpoint instruction (via G_BREAKPOINT()).
So sure enough, the program does get stopped, with gdb reporting:
Program received signal SIGTRAP, Trace/breakpoint trap.
g_object_ref (_object=0x65f090) at gobject.c:2606
2606 old_val = g_atomic_int_exchange_and_add ((int *)&object->ref_count, 1);
(gdb) _
which is a great start. Now, normally I'd script some commands to be run at a breakpoint I manually set using commands 3 (for breakpoint 3, say). But the equivalent for SIGTRAP, namely handle SIGTRAP, doesn't give me the option of doing anything particularly interesting. Is there a good way to do this?
(I'm aware that there are other ways to debug reference leaks, such as setting watchpoints on the object's ref_count field, refdbg, scripting regular breakpoints on g_object_ref() and g_object_unref(). I'm about to go try of those now. I'm looking specifically for a way to script a response to SIGTRAP. It might come in useful in other situations, too, and I'd be surprised if gdb doesn't support this.)
Do you want to show some values and continue execution of the program? In that case, just define a macro that displays the values you're interested in, continues execution and calls itself recursively:
define c
echo do stuff\n
continue
c
end
GDB doesn't support it.
In general, attaching a command script to signal makes little sense -- your program could be receiving SIGTRAP in any number of places, and the command will not know whether a particular SIGTRAP came in in expected context or not.