Start process via SSH and GDB? - linux

This question sounds very stupid to me, but if this is somehow possible it would be really helpful.
My application is crashing and I need to debug it. And I run this application in another computer, via SSH (it's an HTTP server). If I could leave a terminal running the application over GDB and SSH all the time, I'd be able to find the bugs. But I don't have a free computer to do that. What can I do? Is there a way to start GDB with nohup(1) plus &> and stuff like that, so I can see GDB output (where command, for example) later?

A classical Unix program called screen is your friend (or its competitor tmux). It allows to keep a virtual console open across multiple logins:
screen
starts such a session; using you can detach from that; using
screen -r
you can reconnect to it later.
However, you don't even need that; just make your program leave a core dump when it crashes; ulimit -c unlimited says "every program that crashes leaves a core dump"; you can then just open the core dump using gdb later on, and everything will be as if you ran the program inside gdb when it crashed.
gdb core.123456

Related

Resume gdb session after parent shell terminated

I was running gdb over an SSH shell. After a while the shell disconnected due to being idle. On reconnecting I see the gdb instance still running.
How do I take control of the running gdb instance?
Can I start a new gdb instance and take over the session from the running gdb instance?
Note: This is not about keeping the SSH session alive. This is more about taking control of gdb from another shell instance. Regardless of whether it is running from SSH or locally.
You probably can't do anything with the gdb that is already running. It probably no longer has a controlling tty at all. What you probably want to do is kill it and then start a new gdb process, but do it inside a program like tmux or screen. If you do that, then if/when you get disconnected you can easily reattach to the tmux/screen session any time. Just check out the manual for those programs. They do pretty much the same thing. I think tmux is a little more powerful.

Communicating with another terminal application via SSH

I'll start off by saying I'm no Linux guru, or even close. I use it to develop embedded applications and it serves my purposes just fine as such.
I have a program which is running (I suppose the correct terminology is "running in a terminal") on a CentOS 6.3 box. For debug and statistics, I have a routine which monitors the keyboard (stdin file) and spits out items as requested by the given key-presses. That all works fine when I'm sitting in front of it.
But I'd like to be able to perform these simple functions: press a key, see some output, remotely. I can SSH into the box and execute commands, but I can't "see" this program.
I have tried searching "communicating with other terminals using ssh" and myriad variations, but I guess I'm not asking this correctly -- the search results are worthless.
What I'd like to be able to do is login to my account and then somehow "see" my program running and type keys and see its output. Is this a stdin/stdout redirection issue?
I apologize in advance if this is painfully obvious and I'm just an idiot, but I'd still like to know how to do it... :)
If you use the linux 'screen' utility, you can re-attach to the original terminal session that you used to start the program. basically you just type screen and then run your program. here is more useful info on screen:
http://www.rackaid.com/resources/linux-screen-tutorial-and-how-to

What can cause SIGHUP to be generated?

We have about 40 computers running identical hardware and software. They all run Ubuntu 11.10. They all have just one user account to log in. The .profile file is set up to launch a daemon process. The code for the daemon is written in C.
Once in a few weeks, we get a report that the daemon is no longer running. This does not happen on all computers but just one or two. We cannot reproduce the problem consistently.
Looking at the code, the application quits when it receives either SIGHUP or SIGTERM.
As I understand, SIGHUP is generated when a user logs off. In our case, the user never logs off. I am wondering if it is possible that SIGHUP could have been generated for some other reason. Any other thought would be appreciated.
Well, there are a couple of things to note about SIGHUP. Firstly, its origin is from the concept of a hang-up, i.e. loss of connection to a console over something like a modem. In modern parlance this generally means it has lost its controlling tty. Unless you've taken care to detach from your tty, any program started in a given terminal will receive a SIGHUP when the terminal is closed. See here for details on how to do this in your program. Other options include:
running your program inside screen or tmux
run your program with nohup or some other daemonising framework
The other possibility is something is deliberately sending your process a SIGHUP which by "tradition" is often used to signal a process that it should re-read its configuration.
Signals can be sent using kill utility or kill syscall.
Of course, you can try and find out who is sending that signal or disconnecting your terminals or network connections, but there is simpler practical way to fix your problem.
When code is supposed to run as a daemon, but really isn't (just like yours), there is a wrapper that can turn any program into daemon. Surprise - this wrapper is called daemon! It has lots of options, probably most importantly for you, option to automatically restart your utility should it ever die for any reason.
If this command is not installed on your Ubuntu, just sudo apt-get install daemon, and man daemon to get started.

Keeping a X11 application alive, which I can disconnect from and reconnect to afterwards

I currently have a tiny, headless (and I certainly want to keep it that way :) ) Linux Virtual Machine set up with Vagrant and VirtualBox which, for testing, I want to run an X11 application (Firefox) whose output comes to Xming on my real machine. All that's hunky dory, working perfectly, but I'm not happy yet!
What I want to be able to do is do a few setup things, make sure everything's running correctly, then disconnect from the server and let the testing run it's course. If however something goes wrong, or I want to just check the current status of things (some of the tests may run into hours), I'd like to then hop back onto the server and point the X11 output to my machine again. But despite a good deal of Google-ing and learning loads about X11 that I didn't know a few hours ago, I can't find anything about choosing where the output of an X11 application goes, except at startup, ie;
DISPLAY=:10 firefox &
I had read some random blog post that Xephyr XServer did this (kind of act as an intermediate X11 buffer, which then redirects if you want it to, otherwise just outputs to /dev/null), but I can't find any other reference to it, or anything else doing that.
There's a program called Xpra that works sort of like "screen" but for X-sessions. It would start a separate X session from the main one, for the remote access, but you can connect/disconnect to it at will from the host machine.
http://www.xpra.org/
I currently have one acceptable way to do this, which will serve my purpose, I have a vnc4server running that takes firefox's output, and then I can connect and disconnect to that without any issue at all, just like a normal VNC server. This allows me to do what I want to do, but not how I want to do it. I'd like to be able to do this without the need of a VNC server at all.

How to launch firefox from C program/daemon under Linux

I am having some problems with launching firefox from a Linux daemon written in C.
When I launch firefox on the machine itself (via terminal) from the command shell using /usr/bin/firefox it works OK and a firefox browser window lanunches as it should.
However if I try this within my C daemon using system("/usr/bin/firefox"), firefox launches its process in the terminal but the browser window is not opened?
A similar thing happens when I try to do this using remote terminal acces. It's something to do with telling the system to open firefox in window mode rather than trying to open it in terminal mode - but I dont know how to specify this using bash commands?
I am using Lubuntu 11.10 in my Linux System.
Any help is most appreciated.
There's a reason I asked why you're attempting to do what you want. I didn't want to get into great details in my comment.
Firefox on Unix is an X-Window process (most of the Linux/Unix desktops are based upon the X11 protocol which is the heart of X-Window). What X-Window does is separate the display of the program from the process running the program. For example, I am now running Firefox from a Linux box at work, but the Linux box is actually displaying the Firefox browser window at home on my Mac.
In order to do this, I had to:
Run X11 on my Mac. The X11 program creates a default X11 client display called 0.0 which pretty much says the first screen and the first instance of X11 running (computer geeks like counting from zero). The process runs in the background on my Mac. In a certain sense, it's really a server process and not a client because it's waiting on port 6000 for a client X11 process (Firefox) to tell it what to do.
Before I run firefox, I have to tell my Mac's X11 process that I grant the X11 server running Firefox to be able to display on my X11 client process. Otherwise, you can imagine someone spamming another person by continuously popping up Windows on their display. You can use the xhost program to do this.
In order to run Firefox on the Linux box, I have tell the Firefox process what X11 client I'm running it on. I can do this by setting the DISPLAY environment variable to something like "10.0.1.33:0.0". This means the X11 client is running the the machine on IP address 10.0.1.33, and I want you to use the first screen, and the first instance of the X11 client on that screen.
Now, I can run Firefox on my Linux box, and the display will display on my Mac.
The problem you're running into is that there's simply no X11 client when you're starting FireFox as a daemon process. An X11 client is associated with a user and a display of some sort. The display could be a virtual display, but there's got to be an X11 client that's running and is addressable in some way, so the process knows where to display the output.
By the way, you said daemon which has a very, very specific meaning in Unix/Linux. A daemon is a process that runs in the background and usually has a service (and a port) associated with it. For example, there's an FTP daemon called ftpd, the mail server uses the sendmail daemon, ssh has the sshd daemon. Daemons have no display associated with them.
However, it looks like you might be using the word to mean launching Firefox via another process. Is that true? If so, you'll have to make sure that Firefox knows the X11 display to use (there's a command line setting to use to specify the display), and that your X11 client (your login session) has given permission for another process to update your display with the program window.
Can you please explain what you're trying to do in a bit more detail? If you simply want to download a file from a remote http server (which of course is running the http daemon process called httpd), you should use curl or wget which don't require a display and are way simpler to use. If you're trying to do something else, let us know exactly what it is.
Firefox needs to know which display it should open on. When you run it from within a gui, even through a terminal emulator, the DISPLAY environment variable is set to the appropriate value. When you launch from the daemon, try system("/usr/bin/firefox -display=:0").
Make sure that the DISPLAY environment variable is properly set in your daemon to refer to the X server that you want your firefox to use.
If the daemon is run as a different user account than the user account that "owns" the X server that you want to use, you will also need to use xauth(1) to configure the authentication token to grant permission to use the X server.
Often times, it is far easier to use ssh -X to tunnel X and properly configure the xauth(1) tokens in one go than try to manage xauth(1) tokens yourself. Maybe adding ssh -X into your environment would be suitable, maybe not. (I've even used ssh -X root#localhost before when I needed to run an X client as root and didn't want to bother with configuring xauth(1) manually. ssh(1) is just so much easier.)

Resources