How to find out why daemon tools supervise is exiting - node.js

I run a node process (websocket server) on an AWS instance. I used to start it like this:
node websocket/index.js
But have recently swiched to using daemontools supervise to run this process so that it respawns if it should quit or die for any reason.
So, I now run the process like this (from within the dir): supervise . &. The following is my ./run file:
#!/bin/sh
node websocket/index.js
This generally works well. When I manually kill -9 the Node process to test out it out, Supervise respawns it correctly.
However, every morning when I check in on things, the Node process and the Supervise process are both dead and nowhere to be found in ps. I confirmed that the system is not rebooting by looking at uptime.
How can I find out why the Supervise and Node processes are dying overnight? And how can I prevent this?

Update: I switched to using the node module forever to keep my process running, and it has been great so far.
https://www.npmjs.com/package/forever

Related

Does supervisord kill running processes on with the restart command?

We have supervisord running in production and I'd like to know if after running $ supervisorctl restart group-name:* if running processes are killed immediately or if supervisor lets running processes finish.
Tried my best to find that out in the docs and the source code.
As far as I know, supervisorctl will kill process, because usually under supervisor work worker script which never stops, hence it is no way to supervisorctl understand when script is ready to be stopped.

When we exit command prompt when npm start is still running (Node.js), will the process end and the server stop?

I'm new to node and don't really know how things will turn out.
The short answer is: yes, the process will end.
When you run a process in a command prompt (or terminal), then the command processor basically pauses while your process is running. When you shut down the command prompt, you kill whatever process is running in that prompt.
On Linux systems, there's a couple ways of keeping the program around when killing the terminal:
node server.js &
The & tells the terminal to run the job in the background. You'll notice that it returns you immediately to a prompt.
However, when you actually log out, your process will get killed. In order to prevent that, you can use the nohup command (as indicated in the comments by #YTowOnt9):
nohup node server.js &
This tells the process to ignore the HUP (hangup) signal:
From Wikipedia:
nohup is a POSIX command to ignore the HUP (hangup) signal. The HUP signal is, by convention, the way a terminal warns dependent processes of logout.
On Windows system, you can use the START command, but as far as I know there's no way to keep the program running after logging out short of turning the program into a Windows Service (which is a whole other topic):
start node server.js
If you want to prevent the creation of a new window:
start /b node server.js

How do I shutdown a node project started by Forever? ungit

I'm using the open source project called Ungit. It is based on node.js and I really enjoy it but I don't know how to shut it off other than rebooting the whole box.
Ungit is using "forever-monitor" to have a process run forever. So when I do nohup ungit > /dev/null & I can't shut it down by killing it's process as it will restart it.
I have looked up forever-monitor project and I was able to install it globally and try to shut it by running forever stop [number] but forever list shows nothing and I don't know how to shut it down.
Thanks for any advices,
with forever list you can see list of forever process
with forever stop 0 you can stop first process .
with forever stop myscript.js you can stop a process with name of script
find the process that forever-monitor is running under and kill it first then kill the ungit process.
http://www.cyberciti.biz/faq/kill-process-in-linux-or-terminate-a-process-in-unix-or-linux-systems/

how to automatically restart a node server?

We are finishing development of a project, the client is already using it but occasionally some errors occur - crashing the server.
I know I could register a service as 'upstart' script on linux, in order to have my node service restart when it crashes.
But our server is running other stuff, so we can't restart it.
Well, actually, while writing, I realize I have two questions then:
Will 'upstart' work without having to reboot? Something is just whispering yes to me :)
If not, what other option would I have to 'respawn' my node server when it crashes?
Yes, upstart will restart your process without a reboot.
Also, you should look into forever.
PM2 is a Production process manager for Node.js app.
If your focus for automatic restart is an always running application, I suggest to use a process manager. Process manager, in general, handles the node process(es if cluster enabled), and is responsible for the process/es execution. PM leans on the operative system: your node app and the OS are not so strinctly chained because the pm is in the middle.Final trick: put the process manager on upstart. Here is a complete performance improvement path to follow.
Using a shared server and not having root privileges, I can't download or install any of the previously mentioned libraries. What I am able to do is use a simple infinite bash loop to solve my issue. First, I created the file ./startup.sh in the base directory ($ vim startup.sh):
#!/bin/bash
while:
do
node ./dist/sophisticatedPrimate/server/main.js
done
Then I run it with:
$ bash startup.sh
and it works fine. There is a downside to this, which is that is doesn't have a graceful way to end the loop (at least not once I exit the server). What I ended up doing is simply finding the process with:
$ ps aux | grep startup.sh
Then killing it with
$ kill <process id>
example
$ kill 555555

How to run node.js app forever when console is closed?

I connect to my remote server via ssh. Then I start my node.js app with Forever. Everything works fine until I close my console window. How to run node.js app FOREVER on my remote server even when I close my connection via ssh? I just want to start an app and shut down my copmputer. My app should be working in the background on my remote server.
You may also want to consider using the upstart utility. It will allow you to start, stop and restart you node application like a service. Upstart can configured to automatically restart your application if it crashes.
Install upstart:
sudo apt-get install upstart
Create a simple script for your application that will look something like:
#!upstart
description "my app"
start on started mountall
stop on shutdown
# Automatically Respawn:
respawn
respawn limit 99 5
env NODE_ENV=production
exec node /somepath/myapp/app.js >> /var/log/myapp.log 2>&1
Then copy the script file (myapp.conf) to /etc/init and make sure its marked as executable. Your application can then be managed using the following commands:
sudo start myapp
sudo stop myapp
sudo restart myapp
Two answers: One for Windows, one for *nix:
On Windows, you can use the start command to start the process disconnected from your instance of cmd.exe:
start node example.js
On *nix, there are two aspects of this: Disconnecting the process from the console, and making sure it doesn't receive the HUP signal ("hang up"), which most processes (including Node) will respond to by terminating. The former is possibly optional, but the latter is necessary.
Starting disconnected from the console is easy: Usually, you just put an ampersand (&) at the end of the command line:
# Keep reading, don't just grab this and use it
node example.js &
But the above doesn't protect the process from HUP signals. The program may or may not receive HUP when you close the shell (console), depending on a shell option called huponexit. If huponexit is true, the process will receive HUP when the shell exits and will presumably terminate.
huponexit defaults to false on the various Linux variants I've used, and in fact I happily used the above for years until coderjoe and others helped me understand (in a very long comment stream under the answer that may have since been deleted) that I was relying on huponexit being false.
To avoid the possibility that huponexit might be true in your environment, explicitly use nohup. nohup runs the process immune from HUP signals. You use it like this:
nohup node example.js > /dev/null &
or
nohup node example.js > your-desired-filename-or-stream-here &
The redirection is important; if you don't do it, you'll end up with a nohup.out file containing the output from stdout and stderr. (By default, nohup redirects stderr to stdout, and if stdout is outputting to a terminal, it redirects that to nohup.out. nohup also redirects stdin if it's receiving from a terminal, so we don't have to do that. See man nohup or info coreutils 'nohup invocation' for details.)
In general for these things, you want to use a process monitor so that if the process crashes for some reason, the monitor restarts it, but the above does work for simple cases.
I would definitely recommend pm2
npm install -g pm2
To start server: pm2 start [yourServerFile.js]
To stop server: pm2 stop [yourServerFile.js]
Close client and server will run forever....will also restart if app crashes.
Ive been running a node server on Ubuntu for months with zero issues
Always, simple is the best, no need upstart, no need forever, just nohup:
nohup node file.js &
Believe me, I'm running so that for my case!
You could install forever using npm like this:
sudo npm install -g forever
Or as a service:
forever start server.js
Or stop service
forever stop server.js
To list all running processes:
forever list
node expamle.js & for example
In Linux, SSH into your remote server and run
screen
to launch into a new screen.
Finally, type ctrlad to detach the screen session without killing the process.
More info here.
I had similar issue and I think using forever will help to handle crashed and restarts
You can install forever globally:
sudo nom install -g forever
And run this command:
nohup forever server.js &
This should handle all the trouble of closing the terminal, closing ssh session, node crashes and restarts.
If you're running node.js in a production environment, you should consider using PM2, forever.js, or Nodemon.
There is no shortage of articles online comparing the different packages.
This is only a partial answer for Windows. I’ve created a single line Visual Basic Script called app.vbs that will start your node application within a hidden window:
CreateObject("Wscript.Shell").Run "node app.js", 0
To execute it automatically at startup, open the %AppData%\Microsoft\Windows\Start Menu\Programs\Startup\ directory and add a shortcut to the app.vbs file.
More info at: https://keestalkstech.com/2016/07/start-nodejs-app-windowless-windows/
Wow, I just found a very simple solution:
First, start your process (node app)
forever dist/index.js
run: ^Z cmd + z.
Then: bg. Yeah.. bg (background).
And pum.. you are out.
Finish with exitif you are with sshor just close the terminal.
my start.sh file:
#/bin/bash
nohup forever -c php artisan your:command >>storage/logs/yourcommand.log 2>&1 &
There is one important thing only. FIRST COMMAND MUST BE "nohup", second command must be "forever" and "-c" parameter is forever's param, "2>&1 &" area is for "nohup". After running this line then you can logout from your terminal, relogin and run "forever restartall" voilaa... You can restart and you can be sure that if script halts then forever will restart it.
I <3 forever

Resources