I wrote http server in node.js, which is a like reverse-proxy for Amazon S3, and deployed it in production environment with Node's Cluster module, nodejitsu/forever and Nginx.
It worked very good but one day (today) it stopped responding. I checked node's console.log() outputs and processes but I found nothing strange.
Gist of my code is like:
http.createServer(function(webReq, webRes) {
http.get(s3Options, function(s3Res) {
if (s3Res.statusCode == 200) {
s3Res.on('end', function() {
webRes.end('Found data on S3');
});
} else {
webRes.end('No data on S3');
}
}).on('error', function(e) {
console.log('problem with s3Req: ' + e.message);
});
}).listen(1337);
Node processes are all alive (2 child workers) without forever's restarting:
# ps x | grep node
31436 ? Ss 3:43 node /usr/bin/forever -l LOG -o OUT -e ERR -a start server.js
31437 ? Sl 0:10 node /root/server.js
31440 ? Sl 1:17 /usr/bin/nodejs /root/server.js
31441 ? Sl 1:17 /usr/bin/nodejs /root/server.js
Then I doubted too-many-connection stuffs and did "lsof -p PID | wc -l" but the counts were all in good conditions - only dozens.
My node.js experience is only a week or so. Did I miss something important?
Related
I wrote the following web server in node.js using express and hbs that can run shell files, but I keep getting the following error when I type this url into a web browser
linux username here is replaced with my linux username
http://127.0.0.1:3000/run?file="/home/linux username here/nasServer/GameServers/minecraft/1.16.2 server/run.sh"
stderr: bash: <path to .sh file here>: No such file or directory.
child process exited with code 127.
Contents of nodejs file:
const express = require('express')
const hbs = require('hbs')
const app = express()
const port = 3000
// Set up handlebars engine
app.set('view engine', 'hbs')
app.get('/run', (req, res) => {
const { spawn } = require('child_process');
let callback = ""
ls = spawn("bash", [req.query.file])
ls.stdout.on('data', function (data) {
console.log('stdout: ' + data.toString());
});
ls.stderr.on('data', function (data) {
console.log('stderr: ' + data.toString());
});
ls.on('close', function (code) {
console.log('child process exited with code ' + code.toString());
});
res.send({
out: callback
})
})
app.listen(port, () => {
console.log(`App listening on port ${port}`)
})
This is the contents of run.sh:
#!/bin/bash
java -Xmx5G -jar /home/linux username here/nasServer/GameServers/minecraft/1.16.2\ server/spigot-1.16.2.jar nogui
Hello, I don't know why I decided to try to fix this since I know just a little bit about coding (only scripting actually lol) and nothing about nodeJs but I got fun testing your app with the help of my friend google !
First of all, since i dont have your minecraft files (jar etc.), I just writed a little script "test.sh", that will just echo your command:
❯ cat test.sh
#!/bin/bash
echo "java -Xmx5G -jar /home/linux username here/nasServer/GameServers/minecraft/1.16.2\ server/spigot-1.16.2.jar nogui"
second of all after like 2hours of training, modifying, testing by adding/deleting stuff into your app to understand how it works, I finally came back to your original app when i find that it works with this:
http://localhost:3000/run?file=test.sh
here is the browser output (as expected):
{"out":""}
here is the console output:
❯ node 71963151.js
App listening on port 3000
stdout: java -Xmx5G -jar /home/linux username here/nasServer/GameServers/minecraft/1.16.2\ server/spigot-1.16.2.jar nogui
child process exited with code 0
The fact is that when we remove the double quotes from the query it works fine, but when I add back the double quotes like you are trying to do:
http://localhost:3000/run?file="test.sh"
here is the browser output (as expected):
{"out":""}
but here is the consol output:
❯ node 71963151.js
App listening on port 3000
stderr: bash: "test.sh": No such file or directory
child process exited with code 127
So, to conclude, instead of trying to run this on your browser:
http://127.0.0.1:3000/run?file="/home/<linux username here>/nasServer/GameServers/minecraft/<1.16.2 server>/run.sh"
try this:
http://127.0.0.1:3000/run?file=/home/<linux username here>/nasServer/GameServers/minecraft/<1.16.2 server>/run.sh
A lot of documentation helped me out to understand the way nodejs works, I loved doing this :p thank you, You made me want to code!
bguess.
I fixed the problem by replacing all the windows line endings with Unix line endings, then bash found the .sh file.
I have a nodejs application running on a live server. I start a new node process using the following command in Terminal of VSCODE by accessing the server through SSH.
nohup node filename.js &
Mostly I can see the process id using the following command in the VSCODE terminal.
netstat -lpn | grep 30001
This command gives the following output:
tcp6 0 0 :::30001 :::* LISTEN 21552/node
But, sometimes it doesnt show up any process id, as shown in the following output:
tcp6 0 0 :::30001 :::* LISTEN -
In case the process dies to due some technical error, it should get restarted automatically. I have executed the following code through a cron in every 5 minutes for this, which works.
const find = require('find-process');
var spawn = require('child_process').spawn;
find("port", "30001")
.then((list)=> {
console.log("list::", list);
if (!list.length) {
spawn('node', [`${__dirname}/filename.js`], {
detached: true,
stdio: 'ignore'
}).unref();
}
}, function (err) {
console.log(err.stack || err);
});
Following is my cron
*/5 * * * * node path-to-js-file/crontab.js
My Question:
Why my node instance on port 30001 is sometimes not having a pid while the application contained inside it is still accessible?
kill -9 will need a Process Id which I dont have as showed above. How to kill such process through command so that it can be restarted?
To show the proccess pid you can use the process module of nodejs.
var process = require('process');
console.log(`Process pid ${process.pid}`);
Im looking to use ngrok on port 4000 and which is a command that will export a Forwarding URL. Every time this runs theres a new randomly generated URL.
I would like to pass that url http://2e1v870f.ngrok.io to a node process.env variable, rather then hard-coding it evey time.
For example in bash:
ngrok http 4000 | <process/define_something> | FORWARDING={something} node index.js
Plus these are in two running processes I've used npm-run-all to do something like this. https://www.npmjs.com/package/npm-run-all
ngrok by #inconshreveable (Ctrl+C to quit)
Session Status online
Version 2.2.8
Region United States (us)
Web Interface http://127.0.0.1:4041
Forwarding http://2e1v870f.ngrok.io -> localhost:4000
Forwarding https://2e1v870f.ngrok.io -> localhost:4000
Connections ttl opn rt1 rt5 p50 p90
0 0 0.00 0.00 0.00 0.00
I've turned to using the node wrapper for ngrok as I couldn't access the output from bash. Here's an example start.js:
if (!process.env.ROOT_URL) {
var ngrok = require('ngrok');
var shell = require('shelljs');
ngrok.connect(3000, function(err, url) {
shell.exec('ROOT_URL=' + url + ' meteor --settings settings.json', function(code, stdout, stderr) {
console.log('Exit code:', code);
console.log('Program output:', stdout);
console.log('Program stderr:', stderr);
});
});
}
When i run this pidof command by hand, it works. Then put into my server.js.
// send signal to start the install script
var spw = cp.spawn('/sbin/pidof', ['-x', 'wait4signal.py', '|', 'xargs', 'kill', '-USR1']);
spw.stderr.on('data', function(data) {
res.write('----- Install Error !!! -----\n');
res.write(data.toString());
console.log(data.toString());
});
spw.stdout.on('data', function(data) {
res.write('----- Install Data -----\n');
res.write(data.toString());
console.log(data.toString());
});
spw.on('close', function(data) {
res.end('----- Install Finished, please to to status page !!! -----\n');
console.log('88');
});
In the web i only see "----- Install Finished, please to to status page !!!". My install script seems never get this USR1 signal. Anything wrong please ?
The problem is that you have two separate commands. You are piping the output of your /sbin/pidof command to the input stream of your xargs command. If you are using spawn (rather than exec, which a string exactly as you would write on the command line), you need to spawn one process per command.
Spawn your processes like this:
const pidof = spawn('/sbin/pidof', ['-x', 'wait4signal.py']);
const xargs = spawn('xargs', ['kill', '-USR1']);
Now pipe the output of the first process to the input of the second, like so:
pidof.stdout.pipe(xargs.stdin);
Now you can listen to events on your xargs process, like so:
xargs.stdout.on('data', data => {
console.log(data.toString());
});
I have a child process that I am using as follows in node.js. Instead of redirecting the output to the console I would like to put the output in a log file located somewhere on the machine this is running on (and should work for both windows and mac).
The code below is what I am using and I would like to output the files into a log file. What changes needed to do that here? Thanks!
My Code:
var spawn = require('child_process').spawn,
ls = spawn('ls', ['-lh', '/usr']);
ls.stdout.on('data', function (data) {
console.log('stdout: ' + data);
});
ls.stderr.on('data', function (data) {
console.log('stderr: ' + data);
});
ls.on('close', function (code) {
console.log('child process exited with code ' + code);
});
Here's an example of logging to file using streams.
var logStream = fs.createWriteStream('./logFile.log', {flags: 'a'});
var spawn = require('child_process').spawn,
ls = spawn('ls', ['-lh', '/usr']);
ls.stdout.pipe(logStream);
ls.stderr.pipe(logStream);
ls.on('close', function (code) {
console.log('child process exited with code ' + code);
});
There are two ways you can achieve this, one is using
let logConsoleStream = fs.createWriteStream('./logConsoleFile.log', {flags: 'a'});
let logErrorStream = fs.createWriteStream('./logErrorFile.log', {flags: 'a'});
and redirect all logs or errors using this
ls.stdout.pipe(logConsoleStream ); // redirect stdout/logs only
ls.stderr.pipe(logErrorStream ); // redirect error logs only
by separating log files you will have separate files for Error logs and console logs
this is exactly same as generalhenry shared above
And Second Way for Achieving this with the help of Command Line
when you execute node app from the command line
node app/src/index.js
you can specify here where you want to redirect logs and Errors from this application
there are three stream redirection commands using the command line
`>` It will redirect your only stdout or logs to the specified path
`2>` It will redirect your errors logs to the specified path
`2>&1 ` It will redirect all your stdout and stderr to a single file
example: how you will use these commands
node app/src/index.js > ./logsOnly.txt
node app/src/index.js 2> ./ErrorsOnly.txt
node app/src/index.js 2>&1 ./consoleLogsAndErrors.txt
I hope someone coming later finds this helpful
if there is I done wrong way please do let me know it will help me and others
Thanks
If you run your JS script with forever then you have the option to define a log file as parameter which will handle all your console.log messages. Not to mention the benefit of keeping your nodejs app live permanently.
Alternatively try this:
sudo forever start myapp.js 2>&1 /home/someuser/myapp/myapp.log
use forever with below options
forever start -o out.log -e err.log server.js
The best answer was in the comments and is mentioned in a previous question here: stackoverflow.com/questions/2496710/nodejs-write-to-file
It is as follows:
var fs = require('fs');
fs.writeFile("/tmp/test", "Hey there!", function(err) {
if(err) {
console.log(err);
} else {
console.log("The file was saved!");
}
});