I have a Node.JS server running on PM2 that's crashing every once in a while because of a database limit, which I'm working on.
In the meantime, I thought I'd try just setting up a cron job in cpanel to restart the server every hour if it's down.
So I wrote a bash script like the following:
#!/bin/bash
status_code=$(curl --write-out %{http_code} --silent --output /dev/null https://website.com/)
date >> cronlog.txt
if [[ "$status_code" -ne 200 ]] ; then
pkill node
nohup pm2 start bin/www &
echo "Site status $status_code" >> cronlog.txt
echo "Restarting Server" >> cronlog.txt
exit
else
echo "Site fine" >> cronlog.txt
exit 0
fi
Running this from an SSH terminal works perfectly; if the site is down, it'll restart it.
However, once I set up the cron job in cpanel, like so: 0 * * * * /home/acc123/fix.sh, looking at the output of cronlog.txt, I see that the script is definitely running every hour, trying to restart the server - it's just that the server doesn't restart.
A preliminary Google suggested that maybe pm2 wasn't on the path that the cron job runs from, so I modified the script to look like this:
#!/bin/bash
status_code=$(curl --write-out %{http_code} --silent --output /dev/null https://website.com/)
date >> cronlog.txt
if [[ "$status_code" -ne 200 ]] ; then
pkill node
nohup /home/acc123/bin/pm2 start /home/acc123/bin/www &
echo "Site status $status_code" >> cronlog.txt
echo "Restarting Server" >> cronlog.txt
exit
else
echo "Site fine" >> cronlog.txt
exit 0
fi
But nothing changes. Looking at the text file I write to, the script is definitely running every hour, and it's definitely picking up that the site is down, but while the words "Restarting Server" get written to the text file, the server doesn't actually start.
Checking nohup.out confirms that nothing has been written to it, suggesting that somehow the command nohup /home/acc123/bin/pm2 start /home/acc123/bin/www & isn't running correctly.
I'm stumped. Has anyone seen something similar before?
Found it. Looks like node itself also wasn't on the path variable for the cron job. Explicitly specifying where node was fixed the problem.
Related
how to make sure first command finishes and then only execute second command in shell script
#!/bin/sh
echo "Stopping application"
#command to stop application
echo "Starting application"
#command to start application
In above code, I wanted to make sure that command to stop application is finished properly and then only start the application.
How to handle this.
Please note in my case if application is already stopped then command to stop application takes some random time to complete i.e. 20sec, 30 sec .
So adding sleep is not proper way.
Main moto behind script is to restart application.
Considering fact that if application is allready stopped it doesnt work properly.
If application is running then the script works perfect.
You can use the command return code and a condition to do this.
#!/bin/sh
echo "Stopping application"
#command to stop application
rc=$?
# if the stop command was executed successfuly
if [ $rc == 0 ]; then
echo "Starting application"
#command to start application
else
echo "ERROR - return code: $rc"
fi
There are 'exit codes', try this:
ls
...
echo $?
0
than:
ls non_existing_file
ls: cannot access 'non_existing_file': No such file or directory
echo $?
2
This command echo $? prints exit code of previous command, if it's 0 than it's OK, all non 0 codes means some kind of error which is not OK.
We are using linux Servers (CentOS) and one backup server
We only login to prod server if any there is any issue, else we check connection from backup server
We do no have any kind of monitoring tool
I have created a simple bash script on backup server as below
#!/bin/bash
date
cat /tmp/servers.txt | while read output
do
ping -c 1 "$output" > /dev/null
if [ $? -eq 0 ] ; then
echo "Server $output is UP"
else
echo "Server $output is Down"
fi
done
How do we get output of this script after log into this server automatically.
~/.bash_profile runs every time you log in, so that seems like the place you would want to put this code.
I was looking to test a node server in a shared hosting environment.
I'm using an ssh terminal session to test.
The server works fine, but of course you can't leave the terminal session without stopping the server.
Using pm2 (npm package), I'm able to keep the server running, but on exiting the terminal session, the pm2 job quits as well, which stops the server. Curious why that would be.
Tried using a crontab to run a shell script that in turn runs the pm2 which in turn starts the node server. The cron tab runs every minute, but the node server never starts.
The sh script works just fine, pm2 works fine, and the node server works fine.
What doesn't work: keeping the node server running after exiting the terminal session.
Here's the shell script that runs the pm2 to trigger the node server...
ps cax | grep node > /dev/null
if [ $? -eq 0 ]; then
echo "Process running."
else
echo "Process not running."
PATH=$PATH:/usr/local/bin
pm2 start '/path/to/NodeServer.js' --restart-delay=100
fi
If someone else like me uses pm2 for godaddy shared hosting, and uses a modified version of Habib's answer with pm2 instead of node, do not forget to include path to node into PATH variable, as it is required by pm2:
PATH=$PATH:/home/user/.nvm/versions/node/v11.15.0/bin
pm2 ps | grep 'my-app'
if [ $? -eq 0 ]; then
echo "Process running."
else
echo "Process not running."
cd /home/user/public_html/app
pm2 start app.js --name my-app
fi
The script above works for me. Specifying only full path to pm2 was not working, as it looks for node, which was not in PATH in the cron environment.
ps cax | grep node > /dev/null
if [ $? -eq 0 ]; then
echo "Process running."
else
echo "Process not running."
/home/user/.nvm/versions/node/v11.15.0/bin/node /home/user/public_html/node/app.js
fi
I used this in cron job in GoDaddy Shared hosting and it is working for me. Before this, I used node /home/user/public_html/node/app.js in the same script but it throws an error because of path of the node. Still, I can use node in the terminal but in cron job, I have to use full path.
I've an init script (/etc/init.d) that should run a my executable jar file as a serviceat boot. I need that this script is runned by a specified user.
With su & sudo is possibile but it split the process and I don't like this.
There is another way to run this script as limited user?
This is the relevant part of my init script:
#!/bin/bash
APP_NAME="myapp"
APP_HOME=/home/user1/jetty
JAVA_HOME=/opt/local/java/latest
echo "Service $APP_NAME - [$1]"
echo "JAVA_HOME -> $JAVA_HOME"
echo "APP_HOME -> $APP_HOME"
echo "APP_NAME -> $APP_NAME"
function start {
if pkill -0 -f $APP_NAME.jar > /dev/null 2>&1
then
echo "Service [$APP_NAME] is already running. Ignoring startup request."
exit 1
fi
echo "Starting application..."
cd $APP_HOME
nohup $JAVA_HOME/bin/java -jar $APP_HOME/$APP_NAME.jar\
< /dev/null > $APP_HOME/logs/app.log 2>&1 &
}
On Ubuntu you should use the program start-stop-daemon for this. It has options for launching daemons as different users, managing pid files, changing the working directory, and pretty much anything else that is usually needed by init scripts.
I am using an Amazon Linux AMI and doing some custom modifications(added an axis2server, etc) on it and saving it as a new AMI. Now what I want to do is when the AMI boots up, start up axis2server(ie.axis2server should automatically start when the instance boots up). For that I used a init script like below and ran the following command:
chkconfig --add axisservice
But when I launch a new instance from my image, the axis2server is not getting started.
I just only need to execute the script /home/ec2-user/axis2-1.6.1/bin/axis2server.sh at startup. Am I missing anything here?
#! /bin/sh
# Basic support for IRIX style chkconfig
###
# chkconfig: 235 98 55
# description: Manages the services you are controlling with the chkconfig command
###
case "$1" in
start)
echo -n "Starting axisservice"
touch ~/temp.txt
cd /home/ec2-user/axis2-1.6.1/bin
./axis2server.sh &
echo "."
;;
stop)
echo -n "Stopping axisservice"
echo "."
;;
*)
echo "Usage: /sbin/service axisservice {start|stop}"
exit 1
esac
exit 0
I went through https://help.ubuntu.com/community/CloudInit as well and it provides a mechanism called User-Data Scripts, where a user can execute a script when launching the script.
$ euca-run-instances --key mykey --user-data-file myscript.sh ami-axxxx
This is a command line option and what I want is something like when I launch the instance through the UI, the script should be started.Therefore, I think the above option can not be used in my case. Please correct me if I am wrong.
Thanks,
H.
I bet the environment is not set(up correctly). This means that I am guessing that your shell script tries to start another program and it's not to be found.
So at first, I'd adjust the start part of your script (current):
echo -n "Starting axisservice"
touch ~/temp.txt
cd /home/ec2-user/axis2-1.6.1/bin
./axis2server.sh &
echo "."
Edited:
echo -n "Starting axisservice"
touch ~/temp.txt
cd /home/ec2-user/axis2-1.6.1/bin
./axis2server.sh
RETVAL=$?
[ $RETVAL -eq 0 ] && echo Success
[ $RETVAL -ne 0 ] && echo Failure
echo "."
So what did I do?
removed & so script waits for your shell script (axis2server.sh) to complete
checked the return status ($?) of your shell script
Further debugging:
Add set -x to your scripts to enable tracing and log both stderr and stdout.
Questions:
Are you are aware that stop (in your service script) doesn't do anything?
touch ~/temp.txt is that supposed to create /root/temp.txt? (I'm guessing root runs this script.)
If none of my suggestions work, can you share axis2server.sh and paste stderr and stdout?