How to debug a remote linux binary? - linux

Here is the situation:
I've got a linux binary that is crashing. No log files, trace files, etc. I need to be able to attach a debugger to it (I have the source locally) and track down the error.
Whats the easiest, best way to approach this problem?

Remote debugging is rather straightforward: on the target platform, launch the application with GDBserver, while specifying the host and port for listening to an incoming TCP connection:
gdbserver HOST:PORT PROG [ARGS ...]
On the development workstation, launch the cross-target GDB:
powerpc-7450-linux-gnu-gdb PROG
Be sure to specify the non-stripped executable. At the GDB console, type:
target remote HOST:PORT
break main
continue
Remote cross-target debugging with GDB and GDBserver

It looks like gdb supports remote debugging.
I've never used it but http://davis.lbl.gov/Manuals/GDB/gdb_17.html looks like a start.
Basically you run the program under gdbserver on the target machine, and gdb connects (over TCP or serial) to the gdbserver.

Related

Remote debugging nodejs app in Intellij with Docker - port already allocated

I want to start debugging node.js app using Intellij and node.js interpreter running on Docker. While running the app works, when I try to debug I get the error:
Error running 'index.js'
com.github.dockerjava.api.exception.InternalServerErrorException:
{"message":"driver failed programming external connectivity on
endpoint focused_poincare
(a17137973880d1be7c6a74fc142184fdda31e0dec8ebd539b09d9dbe4cf70014):
Error starting userland proxy: Bind for 0.0.0.0:55578 failed: port is
already allocated"}
Remote interpreter was configured acccording to the documentation. I have created a new Node.js Run/Debug configuration and entered the following data:
.
What might be the cause for debugging not working?
I use:
Intellij Idea Ultimate v. 2019.1.4 Preview
Intellij NodeJS plugin v. 191.7479.1, NodeJS remote interpreter plugin v. 191.6014.8 and Docker plugin v. 191.7141.44
Docker Desktop Community v. 2.0.0.3
EDIT: Adressing the comments:
Local debugging works. The file (index.js) that I am trying to run consists only of console.log('Hello world!') so I don't spawn any child processes on my own. My host system has Windows 10 Pro as OS, so for checking the open ports on host system I used netstat -an | find "55578", which returned nothing. Moreover, if I try to run docker manually from the command line, using docker run -it -p 55578:55578 node, everything runs and no error is given.
Also, each time I try remote debugging, the port number given by Intellij in an error message seems to be random high port number. I tried looking for open ports just after getting error message, but never found one that is open with a number reported by Intellij and those indeed appear in the output:
My Run/Debug configuration:
My Docker configuration (I had to check "Expose daemon on tcp://localhost:2375 without TLS" in Docker configuration to make Intellij and Docker play together):
EDIT: When I add --inspect-brk=0.0.0.0:55432 as "Node Parameters" in "Run/Debug Configurations" Intellij windows (per this bug report) the container nad program start, but debugging seems to be no-op (e.g. the program does not stop on breakpoints).
After updating to Intellij v. 2019.2 I was able to get the container debugging to work using the workaround already mentioned in my question.
I have added a parameter --inspect-brk=0.0.0.0:55432 to Node parameters option in Run/Debug configuration (see the picture below) and everything seems to be working, including the breakpoints.

The latest ndk-gdb can not connect to my device

I just get trouble to use ndk-gdb to debug my native application. This is my environment:
OS X El Captain 10.11.5
NDK version r12
SDK version 24
Build tool version 24.0.0
Device Nexus 6P with Android N (NPD35K)
The issue:
When I was going to launch ndk-gdb it always fail with the message
Error: unable to connect to device.
Remote connection closed
However, the gdbserver instance has already been launched since my app is paused and I could see the gdbserver in adb shell ps list.
What I have tried:
Instead of start gdbserver by the ndk-gdb script, I launched the
gdbserver manually by adb run-as and make sure the gdbserver
keeps running
Make a socket forward by adb forward :5039 localfilesystem:/the_gdb_socket_path
Use netstat -na | grep 5309 to make sure the port is being listening
Use telnet localhost 5039 to test socket but get the error Connection closed by foreign host
Is there any better way to identify where the issue cause the remote gdb failed on connection?
In my case, I had to root the phone to fix that connection issue, see:
Does ndk-gdb work under macOS Sierra?

gnome-desktop docker image works with Linux Mint Host but not CentOS?

I'm using this docker imagine from the docker repo: traxtopel/docker-centos-gnome-desktop
https://index.docker.io/u/traxtopel/docker-centos-gnome-desktop/
I have a Linux Mint system with Docker 0.9.0 installed that can run the image and launch vncserver with a gnome-session without any problems.
However, when I run the same image on my CentOS6 system (also docker version 0.9.0) and attempt to launch a gnome-session I get the following:
Failed to start message bus: Failed to open "/etc/selinux/targeted/contexts/dbus_contexts": No such file or directory
EOF in dbus-launch reading address from bus daemon
I also tried to launch a gnome-terminal which works in the Linux Mint host but produces the following on my CentOS6 host.
Failed to get the session bus: /bin/dbus-launch terminated abnormally with the following error: EOF in dbus-launch reading address from bus daemon
Falling back to non-factory mode.
Failed to summon the GConf demon; exiting. Failed to contact configuration server; some possible causes are that you need to enable TCP/IP networking for ORBit, or you have stale NFS locks due to a system crash. See http://projects.gnome.org/gconf/ for information. (Details - 1: Failed to get connection to session: /bin/dbus-launch terminated abnormally with the following error: EOF in dbus-launch reading address from bus daemon
)
Since both docker versions are the same, and they are using the same docker image, my only guess is it has to do with the hosts kernel version or possibly the hosts selinux configuration.
Linux Mint Kernel: 3.8.0-19-generic
CentOS6 Kernel: 2.6.32-431.5.1
Any ideas what is causing this behavior?
Looks like the issue has something to do with selinux. I don't have all the details or a "proper" fix but the Linux Mint system was not running selinux while the CentOS6 system was.
I had initially ruled this out since when selinux is put into permissive mode the same behavior occurs. Disabling selinux entirely and rebooting the host system fixes the issue though.
Trying to determine the root cause, I ran selinux in permissive mode and tailed the /var/log/audit/audit.log file however nothing is written out to the log concerning this issue when gnome-terminal or gnome-session is run from the docker image.
If anyone has any better solutions please post however it seems a solution is to disable selinux on the host system.

Remote debug with gdbServer

1) I start GDB on remote machine by gdb myAppName
2) Then i give run -p portNumber
3) Then i run java application which connect with it on above mentioned port.
Now when i am trying to remote debug via gdbserver then the scenario is
1) I run gdb server on remote machine
gdbserver host:port path to application
Now when i try to connect my java client application to the gdbserver port .It will not be able to connect.
So i am searching of alternative of run -p command of gdb in gdb server
What i was trying was remote debugging of C code via eclipse CDT
Any help would be highly appreciated.
gdbserver can be started attached to a pid on some systems according to the docs using the following syntax which is different than the normal invocation
gdbserver hostname : port --attach pid
or gdb can be started with a program that is subsequently started by the remote machine
gdbserver hostname : port program
and then you can attach to it from another remote gdb using
(gdb) target remote hostname: port
Eclipse would probably do the same thing so long at the remote gdbserver is invoked as above
and then start the remote program using a different port
(gdb) run -p someotherport
This 2nd invocation, from your comments above, I assume you have created some flag to your program that allows it to bind to someotherport

Starting ddd with remote gdbserver

I'm debugging a program that runs on a remote target using ddd ( the remote gdbserver is running on localhost over port 1234 for example, but still acts as remote).
I know you can connect to the gdbserver by opening ddd, then calling target remote localhost:1234 on the gdb command line of ddd, as described in:
http://www.gnu.org/software/ddd/manual/html_mono/ddd.html#Remote%20Program
However, I hate having to call the target remote localhost:1234 command every time I open ddd, and I'd like to set it to connect to the gdbserver automatically with an alias. I tried running ddd --rhost localhost:1234 program, but once ddd launches a window pops stating that "GDB could not be started", and in the shell written:
sh: 1: exec: localhost:1234: not found
Anyone knows how can I attach to the remote gdbserver right on launch?
Tnx!
Choose one of:
ddd --eval-command="target remote localhost:1234"
put target remote localhost:1234 into a file and use --command
to execute it
put target remote localhost:1234 into your .gdbinit

Resources