Tomcat automatic recovery - linux

I wonder if there is a way to detect if a Tomcat Server has been crashed on a LInux machine? How can i
recover it (- start it) automatically?
I can do it through a cron job. But that is not what i Want!
I want to detect immediately the time that Tomcat crashed!
Is there a way?
Any help?
Antonis

Take a look at mysqld_safe. It is mysql's launch script that monitors mysql and can recover from some errors and relaunch mysql again. You might get some hints on how to do the same thing for tomcat.
A link on how the script work is here.

Related

jenkins terminates automatically due to a process "/var/tmp/bbb/bbb"

I am using non-containerized jenkins on server. It gets terminated automatically each time I try to restart it.
Process which is killing jenkins is "/var/tmp/bbb/bbb". This process gets triggered by jenkins and even If I try to kill this process and restart jenkins service, jenkins again trigger this process which eventually happen to be a reason for termination of jenkins.
/var/tmp/bbb/bbb
I've also searched on google but couldn't find anything useful. Please help.
servers htop report here
Check your root crontab.
I found the same process, made investigation. My instance was using for mining with scripts in crontab.
These scripts stop processes, stop all security services and syslog, create new users, find ssh-keys, change iptables, etc.

Can I monitor daemon/service with supervisord?

I have a system-V init service/daemon running my application. I wanted to make sure that my application always runs even with conditions where process/service could crash, machine restart. I know of supervisord which is able to monitor process but I am not sure it can monitor service/daemon ?
Looks like the manual advices against it.
There is an answer to a similar question which provides a workaround.
Anyway, I would try to find a way to have that service stay in the foreground.

Upstart and init.d priority

I use foreman to run my node.js applications on the production servers (ubuntu server 12).
Foreman has a great tool to create scripts for upstart.
The problem is that when I reboot a server, my application (managed by foreman) is launched before redis-server and I've to build some tricks in order to wait for a valid connection.
The ideal solution will be to start redis-server earlier, and hen the node application when all is started.
Boot configuration :
redis-server is launched by /etc/init.d/redis-server and is /etc/rc2.d/S20redis-server
my node application is started with /etc/init/stocks-streamer*.conf files
My question is : how would you change the boot order of my node application?
I want to wait for redis-server before my application starts but when I do this, it doesn't start :
start on (started redis-server)
I imagine that it's because no events are sent from init.d scripts to upstart but perhaps there is a way I don't know ?
Thanks by advance for your help !
Perhaps you should have redis be started by foreman instead, so you can better control all the dependencies of your app.
Or make sure foreman start much later than redis (make sure foreman's link in /etc/rc2.d is listed later than S20*.
One more alternative: have redis server also be started by upstart, this is likely going to help upstart manage the dependencies.
There are explanations on how to do this here: https://gist.github.com/bdotdub/714533
And I suggest using "Start must precede another service" instead (http://upstart.ubuntu.com/cookbook/#start-must-precede-another-service) so that redis gets started when you start your own service.
Since this question has no accepted answer, and given I just had the same problem here, I figured I would provide another solution. The question could be re-stated as:
How do I make an upstart job wait on an init.d script?
As the OP says in the question, it's possible to emit an upstart event when the init.d script is started. This way, the upstart job can have a simple start on started SCRIPT_NAME declaration.
In my case, using a custom CentOS-based distribution, my /etc/rc.d/rc is in charge of executing the sysvinit (init.d) scripts. That script is fully upstart aware, and so emits upstart events for each sysvinit script that is started/stopped.
In other words, the /etc/rc.d/rc script has something like this (simplified to leave the juicy stuff):
for i in /etc/rc$runlevel.d/S* ; do
subsys=${i#/etc/rc$runlevel.d/S??}
initctl emit --quiet starting JOB=$subsys
$i start
initctl emit --quiet started JOB=$subsys
done
I imagine you need to take a look at your scripts and add the event emission where you think it's suitable. In my case, the emission was already there.
You can wait for several events in an upstart job. See this question for how to find out what events are available (I haven't found a better documentation to be honest).
In particular the trick to grep -r emit seems very useful.

Starting Tomcat as a service on LINUX

I have to use Tomcat to deploy my webapplication. I have copied the installable files and was able to install tomcat. I have very limited privilages on the linux system. I need to make tomcat run as a system service. Can anyone help?
However, when I logged out and logged in to the Linux machine, Tomcat was still running. Is it that we need not worry to restart Tomcat at every login. Any help needed
Thanks,
vamsi
Based on what you have mentioned above, looks like tomcat is already running in the background. If you want tomcat to run as a service, use the steps here
http://www.spaceprogram.com/knowledge/tomcat_init_d.html
It is possible that tomcat was started as a background process, which explain why it was still running when you logged off and back in.
The advantage of setting it up as a service is that when the server restarts, upon startup it will also startup tomcat.

How can I write a script to keep HSQLDB running in case the process is killed

I'm running HSQLDB in server mode on a Linux server and finding that it occasionally gets killed. I'd like to be able to detect that it's stopped running and then kick off a process that starts it up again.
The DB isn't running very often, so polling would have to be very frequent--once every five minutes.
Look at Monit:
Monit is a free open source utility for managing and monitoring, processes, files, directories and filesystems on a UNIX system. Monit conducts automatic maintenance and repair and can execute meaningful causal actions in error situations.
If you are using soem type of Debian, you might try installing HSQLDB using "apt-get install hsqldb-server. That will give you a nice install and the ability to start with "/etc/init.d/hsqldb-server start"
This will also take care of restarting it if your machine reboots. If you get everything installed correctly the problem of it getting killed may just go away.
I was running into some weird issues starting and stopping hsqldb, but once I got it installed correctly everything took care of itself.

Resources