gracefully shutdown http server - node.js

I write a http + websocket(socket.io) server, and catch SIGINT to stop it.
But I can't stop it gracefully. eg:
in a konsole window, I typed in:
>
xxx ~$ npm start
info - socket.io started
server started at :80
new web socket connection !
^C ### PRESS CTRL+C HERE
xxx ~$ ^C--- SIGINT RECIVED --- # <-- WTF??!! ( I have already been dropped to shell
xxx ~$ ps -f
UID PID PPID C STIME TTY TIME CMD
GongT 18776 2281 0 17:37 pts/11 00:00:00 /bin/bash
GongT 31365 1 0 21:24 pts/11 00:00:00 node main.js
GongT 31548 18776 0 21:24 pts/11 00:00:00 ps -f
# ^ look at PPID of node process
xxx ~$ kill -SIGINT 31365
--- SIGINT RECIVED ---
xxx ~$ kill -SIGINT 31365
--- SIGINT RECIVED ---
xxx ~$ # now I disconnect from client(web browser)
server stopped. bye~
My signal handler like that:
process.on('SIGINT', function (){
console.log(' --- SIGINT RECIVED --- ');
if(isServerRunning){
server.close();
isServerRunning = false;
setInterval(function(){
if(checkEveryThingFinished()){
console.log('server stopped. bye~');
process.exit();
}
});
}
});
My question is :
why I was dropped to shell without end the process, and how to prevent this?
(just waitting until process.exit())

Related

nodejs how to find the process id

I have a nodejs application running on a live server. I start a new node process using the following command in Terminal of VSCODE by accessing the server through SSH.
nohup node filename.js &
Mostly I can see the process id using the following command in the VSCODE terminal.
netstat -lpn | grep 30001
This command gives the following output:
tcp6 0 0 :::30001 :::* LISTEN 21552/node
But, sometimes it doesnt show up any process id, as shown in the following output:
tcp6 0 0 :::30001 :::* LISTEN -
In case the process dies to due some technical error, it should get restarted automatically. I have executed the following code through a cron in every 5 minutes for this, which works.
const find = require('find-process');
var spawn = require('child_process').spawn;
find("port", "30001")
.then((list)=> {
console.log("list::", list);
if (!list.length) {
spawn('node', [`${__dirname}/filename.js`], {
detached: true,
stdio: 'ignore'
}).unref();
}
}, function (err) {
console.log(err.stack || err);
});
Following is my cron
*/5 * * * * node path-to-js-file/crontab.js
My Question:
Why my node instance on port 30001 is sometimes not having a pid while the application contained inside it is still accessible?
kill -9 will need a Process Id which I dont have as showed above. How to kill such process through command so that it can be restarted?
To show the proccess pid you can use the process module of nodejs.
var process = require('process');
console.log(`Process pid ${process.pid}`);

Unable to restart node server after making a change

Some code changes I made to our test server are not taking effect. I am trying to make sure the node server was properly restarted after the change. Following are the processes with name "node" showing as running -
[root#ip-10-30-30-4 lib]# ps -aux | grep node
root 18643 0.0 0.7 916304 26544 ? Ssl 17:22 0:00 /usr/bin/node /usr/lib/node_modules/forever/bin/monitor app.js
root 21479 0.0 1.7 980380 62528 ? Sl 19:31 0:00 /root/.nvm/v7.2.1/bin/node /usr/local/mvc/MVC2.0/app.js
root 21491 0.0 2.1 972432 78220 ? Sl 19:31 0:00 /usr/bin/node /usr/local/mvc/MVC2.0/app.js
root 21858 0.0 0.0 112652 960 pts/1 S+ 19:48 0:00 grep --color=auto node
root 22515 0.0 0.8 920548 31008 ? Ssl Nov03 0:00 /root/.nvm/v7.2.1/bin/node /usr/lib/node_modules/forever/bin/monitor app.js
I tried killing the node processes running app.js (with IDs 21479 and 21491), excluding the PIDs 18643 and 22515. It seems to be monitoring script which restarts the server as soon as it stops. Here are its contents -
var fs = require('fs'),
path = require('path'),
forever = require(path.resolve(__dirname, '..', 'lib', 'forever')),
started;
//
// ### #function (file, pid)
// #### #file {string} Location of the pid file.
// #### #pid {number} pid to write to disk.
// Write the pidFile to disk for later use
//
function writePid(file, pid) {
fs.writeFileSync(file, pid, 'utf8');
}
//
// ### #function start (options)
// #### #options {Object} Options for the `forever.Monitor` instance.
// Starts the child process and disconnects from the IPC channel.
//
function start(options) {
var script = process.argv[2],
monitor = new forever.Monitor(script, options);
forever.logEvents(monitor);
monitor.start();
monitor.on('start', function () {
//
// This starts an nssocket server, which the forever CLI uses to
// communicate with this monitor process after it's detached.
// Without this, `forever list` won't show the process, even though it
// would still be running in the background unaffected.
//
forever.startServer(monitor);
//
// Disconnect the IPC channel, letting this monitor's parent process know
// that the child has started successfully.
//
process.disconnect();
//
As soon as I kill the process, with the following command -
kill <PID>
I still see the same processes with new process IDs, which confirms that the processes were restarted. However, the changes are still not reflecting. Should I kill the ones with PIDs 18643 and 22515 as well? I am not sure how was it run actually.

PM2 breaks child process bash script

I have a place in my code where I need to spawn a bash script, write to its standard input, and read from its standard output.
I do this using node's child_process.spawn.
Unfortunately, when I run this code under pm2, the bash script hangs forever when calling mkdir.
Is there any way to avoid this issue?
test.js
'use strict';
const child_process = require('child_process');
setInterval(() => {
const process = child_process.spawn('./test.sh');
process.on('exit', () => {
console.log('process exit');
});
process.stdout.on('data', (data) => {
console.log('Output: ' + data.toString('utf8'));
});
}, 1000);
test.sh
#!/usr/bin/env bash
TEMP=$(mktemp -d);
echo "Created directory $TEMP"
rm -rf ${TEMP}
echo "Deleted directory $TEMP"
Expected output
Starting
Spawning test.sh
Output: Created directory /tmp/tmp.I6Buifdmlu
Output: Deleted directory /tmp/tmp.I6Buifdmlu
process exit
Actual output
[STREAMING] Now streaming realtime logs for [test] process
2|test | Spawning test.sh
2|test | Spawning test.sh
Environment
OS: Ubuntu 14.04
Node: 4.7.3
PM2: 2.4.0
NB: I have tested this on Mac OSX and there is no problem

Start Shell Script in Java and destroy all processes on Exit

I need some special kind of setup to controll a LED Wall. Sadly i cant really change the programming language i use. My setup looks like this:
Processing (some crazy java fork...) Sketch starts after Boot Process. The Processing Sketch scans a folder for subfolder (other sketches which can be started and controll the LED Wall) and starts a Webserver. The Server renders a List with all the scanned Folders. On Click the "Webserver" launches the selected Sketch via ProcessBuilder. The Processing Sketch looks like this:
import http.*;
import java.util.*;
import java.lang.*;
SimpleHTTPServer server;
String prog = "";
int ExitValue = 1;
ProcessBuilder preparedsketch;
Process runningsketch;
void setup() {
SimpleHTTPServer.useIndexHtml = false;
server = new SimpleHTTPServer(this);
TemplateFileHandler templateHandler = new ResultFiles("index.ftl");
server.createContext("", templateHandler);
}
class ResultFiles extends TemplateFileHandler {
public ResultFiles(String templateFileName) {
super(templateFileName);
}
void createMap() {
Map<String, String> params = queryToMap();
if (params.containsKey("prog")) {
if (params.get("prog").equals(prog)) {
println("Has not changed");
} else {
println("PrevProcess: " + runningsketch);
if (runningsketch != null) {
println("Killing: " + runningsketch);
runningsketch.destroy();
}
prog = params.get("prog");
try {
runningsketch = new ProcessBuilder("/Users/kessleml/dev/pixelpusher/base/processing-quit.sh", "--sketch=/Users/kessleml/dev/pixelpusher/base/sketches/pixelpusher_colourcycle_halloween", "--run").start();
// runningsketch = new ProcessBuilder("/usr/local/bin/processing-java", "--force", "--sketch=" + sketchPath("sketches/" + prog + "/"), "--no-java", "--run").start();
} catch (IOException ex) {
println(ex);
}
println("ProjChagned: " + prog);
println("NewProcess: " + runningsketch);
}
}
File files = new File(sketchPath("sketches"));
String[] fileslist = files.list();
addVariable("files", fileslist);
addVariable("selectedprog", prog);
}
}
Everything works till now. But of course i want to close a running (and looping) Sketch if i change (click on a other Sketch on the Website). The Problem is:
When i launch a selected Sketch via runninngsketch = new ProcessBuilder("Path/To/ProcessingCLI", "--sketch=Path/To/Selected/Sketch", "--run").start(); more than one process launches. The reason for this, is the ProcessingCLI File:
#!/bin/sh
# Prevents processing-java from stealing focus, see:
# https://github.com/processing/processing/issues/3996.
OPTION_FOR_HEADLESS_RUN=""
for ARG in "$#"
do
if [ "$ARG" = "--build" ]; then
OPTION_FOR_HEADLESS_RUN="-Djava.awt.headless=true"
fi
done
cd "/Applications/Processing.app/Contents/Java" && /Applications/Processing.app/Contents/PlugIns/jdk1.8.0_74.jdk/Contents/Home/jre/bin/java -Djna.nosys=true $OPTION_FOR_HEADLESS_RUN -cp "ant-launcher.jar:ant.jar:core.jar:jna.jar:pde.jar:core/library/core.jar:core/library/gluegen-rt-natives-linux-amd64.jar:core/library/gluegen-rt-natives-linux-armv6hf.jar:core/library/gluegen-rt-natives-linux-i586.jar:core/library/gluegen-rt-natives-macosx-universal.jar:core/library/gluegen-rt-natives-windows-amd64.jar:core/library/gluegen-rt-natives-windows-i586.jar:core/library/gluegen-rt.jar:core/library/jogl-all-natives-linux-amd64.jar:core/library/jogl-all-natives-linux-armv6hf.jar:core/library/jogl-all-natives-linux-i586.jar:core/library/jogl-all-natives-macosx-universal.jar:core/library/jogl-all-natives-windows-amd64.jar:core/library/jogl-all-natives-windows-i586.jar:core/library/jogl-all.jar:modes/java/mode/antlr.jar:modes/java/mode/classpath-explorer-1.0.jar:modes/java/mode/com.ibm.icu.jar:modes/java/mode/JavaMode.jar:modes/java/mode/jdi.jar:modes/java/mode/jdimodel.jar:modes/java/mode/jdtCompilerAdapter.jar:modes/java/mode/jsoup-1.7.1.jar:modes/java/mode/org.eclipse.core.contenttype.jar:modes/java/mode/org.eclipse.core.jobs.jar:modes/java/mode/org.eclipse.core.resources.jar:modes/java/mode/org.eclipse.core.runtime.jar:modes/java/mode/org.eclipse.equinox.common.jar:modes/java/mode/org.eclipse.equinox.preferences.jar:modes/java/mode/org.eclipse.jdt.core.jar:modes/java/mode/org.eclipse.osgi.jar:modes/java/mode/org.eclipse.text.jar:modes/java/mode/org.netbeans.swing.outline.jar" processing.mode.java.Commander "$#"
So the ProcessBuilder starts three processes: One sh-process which launches two Java-Children-Processes. When i use runningsketch.destroy() it only kills the sh process. The two Java-processes continue running. (Not sure if this is also because of this bug: http://bugs.java.com/bugdatabase/view_bug.do?bug_id=4770092 since i am developing on MacOS Yosemite. The final Product should run on a Linux Machine.)
My solution was to write a new sh-script which kills all of its children via trap:
#!/bin/sh
OPTION_FOR_HEADLESS_RUN=""
function killAllChildren {
kill -9 -$(ps -o pgid= $$ | grep -o '[0-9]*')
}
trap killAllChildren SIGTERM SIGKILL
# trap "trap - SIGTERM && kill -- $$" SIGINT SIGTERM EXIT
cd "/Applications/Processing.app/Contents/Java"
/Applications/Processing.app/Contents/PlugIns/jdk1.8.0_74.jdk/Contents/Home/jre/bin/java -Djna.nosys=true $OPTION_FOR_HEADLESS_RUN -cp "ant-launcher.jar:ant.jar:core.jar:jna.jar:pde.jar:core/library/core.jar:core/library/gluegen-rt-natives-linux-amd64.jar:core/library/gluegen-rt-natives-linux-armv6hf.jar:core/library/gluegen-rt-natives-linux-i586.jar:core/library/gluegen-rt-natives-macosx-universal.jar:core/library/gluegen-rt-natives-windows-amd64.jar:core/library/gluegen-rt-natives-windows-i586.jar:core/library/gluegen-rt.jar:core/library/jogl-all-natives-linux-amd64.jar:core/library/jogl-all-natives-linux-armv6hf.jar:core/library/jogl-all-natives-linux-i586.jar:core/library/jogl-all-natives-macosx-universal.jar:core/library/jogl-all-natives-windows-amd64.jar:core/library/jogl-all-natives-windows-i586.jar:core/library/jogl-all.jar:modes/java/mode/antlr.jar:modes/java/mode/classpath-explorer-1.0.jar:modes/java/mode/com.ibm.icu.jar:modes/java/mode/JavaMode.jar:modes/java/mode/jdi.jar:modes/java/mode/jdimodel.jar:modes/java/mode/jdtCompilerAdapter.jar:modes/java/mode/jsoup-1.7.1.jar:modes/java/mode/org.eclipse.core.contenttype.jar:modes/java/mode/org.eclipse.core.jobs.jar:modes/java/mode/org.eclipse.core.resources.jar:modes/java/mode/org.eclipse.core.runtime.jar:modes/java/mode/org.eclipse.equinox.common.jar:modes/java/mode/org.eclipse.equinox.preferences.jar:modes/java/mode/org.eclipse.jdt.core.jar:modes/java/mode/org.eclipse.osgi.jar:modes/java/mode/org.eclipse.text.jar:modes/java/mode/org.netbeans.swing.outline.jar" processing.mode.java.Commander "$#"
But somehow, also this doesnt work. Even with starting the new sh-script and sending for example a SIGTERM to the started sh-process, doesnt destroy the two Java-Processes neither the sh-process.
I found the solution:
Java sends a SIGTERM signal, so i had to trap this signal. But the java/processing file wasnt the problem, the sh-script didn work as intended.
I had to add & wait to the and of my script. Otherwise SIGTERM cant be trapped (see this post: https://apple.stackexchange.com/questions/123631/why-does-a-shell-script-trapping-sigterm-work-when-run-manually-but-not-when-ru).
Also the killing process didn work out right. I have to kill all children, the sh-script itself BUT not the parent-processes of the sh-script (in this use case the webserver etc.). So i wrote a function to find all children processes and kill them. Things like kill -9 -$(ps -o pgid= $$ | grep -o '[0-9]*') didnt work since they killed the whole tree. In the end the sh-file looks like this:
#!/bin/sh
function killAllChildren {
getChild $$
pkill -TERM -P $$
}
function getChild() {
cpids=`pgrep -P $1|xargs`
for cpid in $cpids;
do
kill -15 $cpid
getChild $cpid
done
}
trap killAllChildren SIGUSR1 SIGTERM SIGKILL EXIT
cd "/Applications/Processing.app/Contents/Java"
/Applications/Processing.app/Contents/PlugIns/jdk1.8.0_74.jdk/Contents/Home/jre/bin/java -Djna.nosys=true -cp "ant-launcher.jar:ant.jar:core.jar:jna.jar:pde.jar:core/library/core.jar:core/library/gluegen-rt-natives-linux-amd64.jar:core/library/gluegen-rt-natives-linux-armv6hf.jar:core/library/gluegen-rt-natives-linux-i586.jar:core/library/gluegen-rt-natives-macosx-universal.jar:core/library/gluegen-rt-natives-windows-amd64.jar:core/library/gluegen-rt-natives-windows-i586.jar:core/library/gluegen-rt.jar:core/library/jogl-all-natives-linux-amd64.jar:core/library/jogl-all-natives-linux-armv6hf.jar:core/library/jogl-all-natives-linux-i586.jar:core/library/jogl-all-natives-macosx-universal.jar:core/library/jogl-all-natives-windows-amd64.jar:core/library/jogl-all-natives-windows-i586.jar:core/library/jogl-all.jar:modes/java/mode/antlr.jar:modes/java/mode/classpath-explorer-1.0.jar:modes/java/mode/com.ibm.icu.jar:modes/java/mode/JavaMode.jar:modes/java/mode/jdi.jar:modes/java/mode/jdimodel.jar:modes/java/mode/jdtCompilerAdapter.jar:modes/java/mode/jsoup-1.7.1.jar:modes/java/mode/org.eclipse.core.contenttype.jar:modes/java/mode/org.eclipse.core.jobs.jar:modes/java/mode/org.eclipse.core.resources.jar:modes/java/mode/org.eclipse.core.runtime.jar:modes/java/mode/org.eclipse.equinox.common.jar:modes/java/mode/org.eclipse.equinox.preferences.jar:modes/java/mode/org.eclipse.jdt.core.jar:modes/java/mode/org.eclipse.osgi.jar:modes/java/mode/org.eclipse.text.jar:modes/java/mode/org.netbeans.swing.outline.jar" processing.mode.java.Commander "$#" & wait

My node.js doesn't respond without reporting any errors

I wrote http server in node.js, which is a like reverse-proxy for Amazon S3, and deployed it in production environment with Node's Cluster module, nodejitsu/forever and Nginx.
It worked very good but one day (today) it stopped responding. I checked node's console.log() outputs and processes but I found nothing strange.
Gist of my code is like:
http.createServer(function(webReq, webRes) {
http.get(s3Options, function(s3Res) {
if (s3Res.statusCode == 200) {
s3Res.on('end', function() {
webRes.end('Found data on S3');
});
} else {
webRes.end('No data on S3');
}
}).on('error', function(e) {
console.log('problem with s3Req: ' + e.message);
});
}).listen(1337);
Node processes are all alive (2 child workers) without forever's restarting:
# ps x | grep node
31436 ? Ss 3:43 node /usr/bin/forever -l LOG -o OUT -e ERR -a start server.js
31437 ? Sl 0:10 node /root/server.js
31440 ? Sl 1:17 /usr/bin/nodejs /root/server.js
31441 ? Sl 1:17 /usr/bin/nodejs /root/server.js
Then I doubted too-many-connection stuffs and did "lsof -p PID | wc -l" but the counts were all in good conditions - only dozens.
My node.js experience is only a week or so. Did I miss something important?

Resources