GDB not breaking on SIGSEGV - linux

I'm trying to debug an application for an ARM processor from my x86 box. I some followed instructions from someone that came before on getting a development environment setup. I've got a version of gdbserver that has been cross-compiled for the ARM processor and appears to allow me to connect to it via my ARM-aware gdb on my box.
I'm expecting that when the process I've got gdb attached to crashes (from a SIGSEGV or similar) it will break so that I can check out the call stack.
Is that a poor assumption? I'm new to the ARM world and cross-compiling things, is there possibly a good resource to get started on this stuff that I'm missing?

It depends on the target system (the one which uses an ARM processor). Some embedded systems detect invalid memory accesses (e.g. dereferencing NULL) but react with unconditional, uncatchable system termination (I have done development on such a system). What kind of OS is the target system running ?

So i assume that the gdb client is able to connect to gdbserver and you are able to put the break point on the running process right?
If all the above steps are successful then you should put the break point before the instruction which crashes, lets say if you dont know where is it crashing then i would say once the application is crashed, the core will be generated, take that core from the board. Then compile the source code again with debug option using -g option(if binaries are stripped) and do the offline ananlysis of core. something like below
gdb binary-name core_file
Then once you get gdb prompt ,give below commands
gdb thread apply all bt
The above command will give you the complete backtrace of all the threads, remember that binaries should not be stripped and the proper path of all the source code and shared lib should be available.
you can switch between threads using below command on gdb prompt
gdb thread thread_number
If the core file is not getting generated on the board then try below command on board before executing the application
ulimit -c unlimited

Related

Using GDB to Debug VM Running on XEN Hypervisor

I always used GDB to do kernel debug. Normally I would run the target kernel inside a VM and run GDB from the host machine. Currently I am working on a project where I need to use XEN hypervisor and I am in a situation where I feel the need to debug a DOM-U from DOM-0. After some searching I couldn't find a way to use GDB in this scenario. Is it possible to use GDB to debug a guest VM running in XEN? If not, what other alternatives I could use?
You can create a pipe (for example, /tmp/mydebug.pipe), then have this line in the .conf file for your domU:
serial = 'pipe:/tmp/mydebug.pipe'
Then you can simply use a tool such as WinDBG from another domU. See mkfifo.
Other possibly useful links:
https://xenserver.org/partners/developing-products-for-xenserver/18-sdk-development/135-xs-dev-windbg.html
http://www-archive.xenproject.org/files/xensummit_intel09/xen-debugging.pdf
https://www.slideshare.net/xen_com_mgr/from-printk-to-qemu-xenlinux-kernel-debugging

How they do debugging Linux Kernel Core

Now a days debugging become so advanced that even 'core kernel source code' can be debugged using Virtual environment.
But after reading couple of blog related to Kernel Core development it was not clear whether they are debugging using Virtual environment.
They have mentioned that they rely on 'Printing message' rather than using debugging tool, at-least for core component.
So, I Request from 'Linux Kernel Experts' to let me know what is good practice followed while debugging Kernel?
I've tried multiple approaches when trying to debug the kernel.
Sometimes, the easiest way is to just add a few printk statements based on my own conditional values, monitor the serial log and see what's going on. Its especially useful when the function in question is invoked quite often, but you are interested only in a subset of those.
QEMU GDB debugging. I have a buildroot filesystem setup. This means the kernel is lean and it boots up real fast. I start qemu with the -s -S flags, and attach gdb as target remote :1234. Additionally, there aren't very many userspace processes in this setup so its easier to debug the kernel.
VMWare stub. Assuming you are running an Ubuntu VM, it is possible that you can attach gdb to a VMware stub and debug the kernel. Personally, I never have had to pursue this route, but I look forward to trying it out someday.
If you have a kernel for a device that gets stuck in a bootloop and it does not print out any debug information out onto serial, it still might be helpful to try and boot it up using QEMU. Sure, the booting up will probably fail as the kernel tries to load up drivers, but you should be able to attach gdb, get a stack trace and see what the root cause is(perhaps a recursive call).

How to debug windows executable in Linux

I am using Ubuntu 13.10. Is it possible to debug a windows executable under Linux?
I've tried gdb, but it always throw me this error message.
/bin/bash: line 0: exec: /some.exe: cannot execute: Permission denied
I don't have much experience with this but, I think, what you're looking for is winedbg. Initially, it should allow you to debug Win32 applications in Linux.
I suggest you try the IDA Debugger (cross platform debugger).Hope it helps.
You are trying to execute a program designed to run in a OS, from within another one, which could be using a complete different processor than a x86.
In order for a executable program to be run in a specific OS, these modules or services must exist:
A module that can understand the binary code of the executable program and translate, if needed, into binary code of the host processor.
A service that can intercept any instruction identified as a system call, and emulate its behaviour using host resources
A module that is able to load, parse and prepare a process block to acommodate the new "guest" process
This is just for executing a program. To debug it, you also need a service that can interact with the above mentioned modules/services in order to control execution of that program.
That said, you can probably debug an EXE file using some utility from the Wine project, asumming your Ubuntu is running on a x86 processor. Take a look at it.
http://www.winehq.org/
Also, I recall VirtualBox offers some sort of debugging help for processes running on the guest OS, but I'm not sure about this.
No, it is NOT possible to debug a Windows executable under Linux.

core dump not generated

I am working on a PC running CentOS as its operating system.
I also work on a embedded with the same OS.
On my PC, I succeeded to create a core dump file on segmentation fault by changing:
core pattern
core_uses_pid
ulimit -c unlimited
sysctl -p
But on the embedded system nothing works - the core dump is not generated! What could be the reason?
If it matters, the application that I would like a dump of is written in C++.
What can I do to get a core dump on the embedded system?
I've made a little crash program, and core dump is generated from the crash program but not for the one i need !!!
So the problem is not on the O.S, but with the specific program.
I discover that we strip -g executable/library files before sending them to the embeded system.I did the same for my crash program, and this one still produce core dump.
Are you certain the kernel on your embedded system supports core dumps? The feature can be disabled in the kernel build (ref), in which case you may have to fake it yourself using something like google-coredumper.
Ok,
i've made a little mistake when i checked ther program on my computer, i've checked it with another signal than on the embeded system. There were still a problem why for the custom signal handler, ther is no core dump.
Solution is in one of the links:
Unfortunately, if your application is equipped with a customized signal handler, no core dump will be generated, because it is generated only by the default signal handlers. In case your application has a custom signal handler, disable it before starting to debug, otherwise, no core dump will be generated. Some sources in the Internet mention that restoring the default signal handler inside the signal handler after the exception has occurred, and sending it again in a loopback can trigger a core dump. In the tests I did, it did generate a core dump, but the only thing I saw in the core dump was the code that my handler executed (i.e. the calls to signal and kill), so this did not help me. Perhaps on other platforms this trick works better.
On my platform it do work - another solution would to generate the core dump in signal handler. I hears that gcore can do it, with windows core i got an error incompatibility.
I've seen two sources of possible information, both of which point to the /etc/security/limits.conf file:
Linux Disable Core Dumps - Yes, I know you want to enable core dumps, but this could help in reverse
CentOS enabling core dumps - Another source pointing at limits.conf.

GDB shows the wrong thread in postmortem analysis

I am experiencing a strange behavior of GDB. When running a post-mortem analysis of a core, dumped from a heavily multithreaded application in c++, the debugger commands
bt
where
thread info
never tell me the thread which the program actually crashed. It keeps showing me the thread number 1. As I am used to see this working from other Systems, I am curious if is is a Bug in GDB or if they changed the behavior somehow. Can anyone point me to a solution of this, it is PITA to search through 75 Threads, just to find out something the Debugger already knows.
By the way, I am on Debian Squeeze (6.0.1), the version of GDB is 7.0.1-debian, the System is x86 and completely 32-Bit. On my older Debian (5.x) installation, debugging a core, dumped by the exact same source, delivers me a backtrace of the correct thread, as does GDB on a Ubuntu 10.04 installation.
Thanks!
GDB does not know which thread caused the crash, and simply shows the first thread that it sees in the core.
The Linux kernel usually dumps the faulting thread first, and that is why on most systems you end up in exactly the correct thread once you load core into GDB.
I've never seen a kernel where this was broken, but I've never used Debian 6 either.
My guess would be that this was broken, and then got fixed, and Debian 6 shipped with a broken kernel.
You could try upgrading the kernel on your Debian 6 machine to match e.g. your Ubuntu 10.04, and see if the problem disappears.
Alternatively, Google user-space coredumper does it correctly. You can link it in, and call it from SIGSEGV handler.

Resources