Am I setting up domain correctly for nodejs? - node.js

I want to prevent my NodeJS app from crashing and read that using domain is the way to do it. I'm still a little confused though but followed what I saw to get it set up. Is this set up correctly? Do I need to change anything? Thanks.
var express = require('express');
var http = require('http');
var path = require('path');
var d = require('domain').create();
var app = express();
app.configure(function() {
//set up express
app.set('port', process.env.PORT || 3000);
app.use(app.router);
app.use(function(req,res){
res.redirect('/error');
});
});
//launch
d.on('error', function(er) {
console.log('Error!', er.message);
});
d.run(function() {
http.createServer(app).listen(app.get('port'), function(){
console.log('Express server listening on port ' + app.get('port'));
});
});
When I create in error in one of my route files it seems to work correctly, but the error isn't logging. Should I have it log to a file or something so I can check for errors later on?

Well, what you're doing is a bad practice according to the NodeJS documentation, since you can start a memory leak every time the error occurs. There's an example on the domain api docs page of the right way to do it. I'm not sure why console.log isn't working, however -- a log file is probably a good idea. Also be aware that the api for domain is still unstable and may change.
In short you should handle your errors gracefully and start a new worker process when an unexpected an error occurs (using the cluster module to start and stop workers that run into problems).
From the docs, this is what you're not supposed to do, which looks awfully close to what you're doing:
// XXX WARNING! BAD IDEA!
var d = require('domain').create();
d.on('error', function(er) {
// The error won't crash the process, but what it does is worse!
// Though we've prevented abrupt process restarting, we are leaking
// resources like crazy if this ever happens.
// This is no better than process.on('uncaughtException')!
console.log('error, but oh well', er.message);
});
d.run(function() {
require('http').createServer(function(req, res) {
handleRequest(req, res);
}).listen(PORT);
});

Related

How does Sticky session works?

I have this code(Copied from github) which create four(no of cpu cores) child instance of master.I have used sticky session for my socket connection to connect properly to backend and redis adapter message broker.But I cant understand how it is working.
Can someone please explain line line by what is happening
http = require('http'),
express = require('express'),
socketIO = require('socket.io'),
cluster = require('cluster'),
port = process.env.PORT || 3000;
socket_redis=require('socket.io-redis')
var app = express(), io;
server = http.Server(app);
app.set('view engine','ejs');
app.get('/', function (req, res) {
console.log('send message by worker: ' + cluster.worker.id);
res.render('abhi');
});
io = socketIO(server);
io.adapter(socket_redis({host:'localhost',port:'6379'}))
io.on("connection",s=>{
s.on("new",data=>{
console.log("hey you")
})
})
// Add your socket.IO connection logic here
if(!sticky.listen(server,port))
{
server.once('listening', function() {
console.log('Server started on port '+port);
});
if (cluster.isMaster) {
console.log('Master server started on port '+port);
}
}
else {
console.log('- Child server started on port '+port+' case worker id='+cluster.worker.id);
}
It's hard to explain sticky session based on your code you provided. But, in a short explanation; I believe socket.io keeps the connected sockets as a thread/object in it's thread. So when you send a packet/request to a socket(.io) server that packet/request has to reach to the process/server that you made your handshake. Or else it'll most likely fail.
If you are looking for a sticky session and clustering I'd advise you to check out socket.io-redis, it makes things quite easy and smooth.
And to run your nodejs process many times to utilize your entire cpu, you can either use Docker and summon containers for each thread/cpu depending on your setup(which I did for a while ago and they seemed quite good) or you can use nodejs' cluster library.

Issues with Socket.IO with Chrome and Firefox

Another update:
The problem occurs when running on localhost as well. Since I figured out the problem comes from the proxy server, here's its code :
var serverBouncer = bouncy(function(req, res, bounce) {
var path = req.url;
var url = req.headers.host;
if (typeof url !== "string")
return;
var urlArray = url.split('.');
var bouncePort = port;
if (!isNaN(urlArray[0]))
bouncePort = parseInt(urlArray[0]);
else if (String(urlArray[0]).toLowerCase() === "www" && !isNaN(urlArray[1]))
bouncePort = parseInt(urlArray[1]);
bounce(bouncePort);
});
serverBouncer.listen(80);
Update:
I found where the problem came from!!! But I still need to find the solution... There seems to be issues with using newer versions of Socket.io (>= 1.0) with a proxy server (bouncy, in my case).
I recently updated Socket.IO from v0.9.16 to v1.4.5, as well as adding Express to the mix. However, now I cannot open multiple (number seems to vary) tabs in Chrome and Firefox without experiencing strange issues (Edge is the only one to work well). It either hangs, or partially loads html and other resources before it hangs.
After waiting, I often get the error :
Failed to load resource: the server responded with a status of 400 (Bad Request)
When I close one of the tab that's been hanging, it unblocks the other tabs that were also hanging.
The issues were not present before going through with the changes listed above.
I've been doing research for 2 full days and just now decided to post this, as I know it's very vague and I'm probably not providing enough information. As much as I'd like to, it would take a very long time to remember and list everything I tried during that time.
Using Windows 10 with Chrome v51.0.2704.103, Firefox v43.0.1. The server (CentOS) is using node v6.2.2 with mainly the following modules :
express#4.14.0
npm#3.9.5
socket.io#1.4.5
Here's some relevant server code :
var port = 8502;
var socketio = require('socket.io');
var express = require("express");
var http = require('http');
var app = express();
var server = http.createServer(app);
var io = socketio.listen(server);
server.listen(port);
app.get('/', function(req, res, next) {
//Returning index.html
});
io.on("connection", function(socket) {
//Some events...
});
Here's a bit of the client code :
var client = io.connect();
client.on('connect', function() {
//Some events
})
your binding before the server is listening, try something like this
var app = express();
server = app.listen(PORT, function () {
console.log('Example app listening on port ' + PORT + '!');
});
io.listen(server);
I managed to replace the bouncy module with nginx. See my other question for the solution.
nginx : redirect to port according to domain prefix (dynamically)

How to Properly Use Socket.IO in Node.JS for Use in Routes / with Heroku?

Please excuse any noobiness, I'm learning. :)
I have Socket.IO set up so that I can use io.sockets.emit inside of my routes, and I have that working. There are a few problems.
(SOLVED? SEE EDIT 3) To use, I cannot start with the word socket. I have to start with ioor I get "ReferenceError: socket is not defined." I'd like to be able to use socket.broadcast.emit to emit the event to all clients except for the current user. Right now I'm having to do a check on the client side to not execute the event if it's the current user and it's becoming a real headache as I'm having to emit more events as my project progresses.
(SOLVED, SEE EDIT 1 & 2) I have to run the application with node app.js and restart the server manually every time I make a server-side change. When I run nodemon, I get "Port 3000 is already in use." I feel that this must be related to the following...
(SOLVED, SEE EDIT 2) When pushing to Heroku, I have the port from the code below changed from 3000 to 80 in bin/www and app.js, but it does not work (I can see a 404 error for sockets in the console). If this and #2 are caused by dealing with http/ports in both places, how do I properly set this up and why does node app.js work?
I only need to run Socket.IO on the route shown below (battlefield). Am I already doing this with am I already doing this with require('./routes/battlefield')(io)?
bin/www
var app = require('../app');
var port = normalizePort(process.env.PORT || '3000');
app.set('port', port);
var server = http.createServer(app);
server.listen(port);
app.js
var app = express();
var http = require('http').Server(app);
http.listen(3000);
var io = require('socket.io')(http);
app.set('socketio', io);
var battlefield = require('./routes/battlefield')(io);
battlefield.js
var express = require('express');
var router = express.Router();
var returnRouter = function(io) {
router.get('/', function(req, res, next) {
// other stuff
io.sockets.emit('message', 'This works');
socket.broadcast.emit('message', 'Socket is undefined');
})
return router;
};
module.exports = returnRouter;
I tried wrapping my routes in io.on('connection', function (socket) { to be able to use socket, and instead of 'Socket is undefined,' the event does not occur.
var returnRouter = function(io) {
io.on('connection', function (socket) {
router.get('/', function(req, res, next) {
// other stuff
socket.emit('message', 'This is never emitted');
})
})
return router;
};
I apologize for such a lengthy question. THANK YOU for any help! 💜
EDIT1: Writing out this question helped me understand the problem better. I commented out server.listen(port); in my bin/www and nodemon now works. However, the app crashes on Heroku. My Procfile is web: node ./bin/www... does that need to be changed?
EDIT2: After figuring out Edit1 and a bit of Googling, I found that I can't have server.listen(); (bin/www) and http.listen(3000); (app.js).
In bin/www, I removed server.listen();.
In app.js, for clarity's sake I changed var http = ... to var server = ... and had it listen for process.env.PORT || '3000';, taken from bin/www. I also removed app.set('socketio', io); because it looks like that was doing nothing... I wonder why it was in there.
app.js
var app = require('express')();
var server = require('http').Server(app);
var io = require('socket.io')(server);
var port = process.env.PORT || '3000';
server.listen(port);
This also makes Heroku work because of process.env.PORT, hurray! I'm guessing node app.js worked because I was initializing the app with app.js, I guess bin/www is not executed when you do that?
I still need help with #1 (using socket.broadcast.emit) 😇.
EDIT 3: Well, it took me literally the entire day but I believe I have it figured out with one quirk. Of course I couldn't use socket, it is a parameter given on connect. I also need to access socket across different routes and found this SO question. This is what I ended up doing in battlefield.js:
var returnRouter = function(io) {
var socket;
router.get('/', authenticatedUser, function(req, res, next) {
io.on('connection', function(client){
socket = client;
});
// other stuff
res.render('battlefield', {/* data */});
setTimeout(function(){socket.emit('test', 'It works!')}, 500);
});
router.post('/', function(req, res, next) {
// socket can be accessed
});
return router;
};
module.exports = returnRouter;
(Note: I took out a lot of my own code so I don't know if this is copy and pasteable ready, and you should probably check that socket is not null)
Without setTimeout, socket is undefined on GET '/'. To my understanding, the page must render first... Strange that 200 sometimes doesn't work for me and 500 does. I can leave it at 500, but this is for a game so time is pretty important.
My questions now are:
Can this be improved / is there a way I can do this without setTimeout? Am I 'connecting' clients properly with this code and am I (question #4 up there^) using Socket.IO efficiently?
P.S. If no one answers ^ these questions, I'll edit this, answer the question, and accept my answer as best answer.
When you use sockets when doing routing in Node its not that useful.
When ever you navigate to a different name space (eg www.example.com --> www.example.com/some-name-space) your front end variables are deleted and you need to resend them. This works great if you pass an object along with the GET request for that name space. But it doesn't need sockets.
Its done like this on your router file
var canAlsoBePassed = {some: "things"};
router.get('/', function(req, res, next) {
res.render('index', { items: "Can be passed directly", variables: canAlsoBePassed });
});
For sockets the best kind of applications are for single page apps or to replace AJAX requests. Another great thing sockets allows is for the server to be able to push information without the client asking for it.
To answer your question about SetTimeout, no you dont need this.
Make sure the socket script running on your client side is waiting for the document to be loaded.
$(document).ready(function() {
When an io.on('connection' event fires on your server side you know you have a new client to serve.
emit an event from the server side something like a welcome event that makes the client join a specific room. Once you have them in that room you can be listening for any events emitted to that room.
See socket.io official info
Custom namespaces
To set up a custom namespace, you can call the of function on the server-side:
var nsp = io.of('/my-namespace');
nsp.on('connection', function(socket){
console.log('someone connected'):
});
nsp.emit('hi', 'everyone!');
On the client side, you tell Socket.IO client to connect to that namespace:
var socket = io('/my-namespace');
Might not be the most accurate answer to your questions but I hope it pushes you in the right direction.

nodejs - running multiple child_process, but some requests got closed unexpectly

On the server side, I have this nodejs code. I simplified the code here to make the question clear:
var express = require('express');
var app = express();
var spawn = require('child_process').spawn;
app.get('/print_N', function(req, res) {
var child = spawn('python', ['some_tick_data_process.py']);
req.on('close', function() {
console.log('req to close with pid=' + child.pid);
child.kill('SIGTERM');
});
child.stdout.pipe(res);
child.stdout.on('error', function(err) {
console.log(child.pid + " error !!!")
console.log(err);
});
});
app.listen(3000);
console.log('Listening on port 3000...');
The underlying some_tick_data_process.py is quite I/O intensive. On the client side, I have a small python application to read the stream. The problem is that some processes will run into error "req to close". With a small number of processes, it is OK. I tried:
var http = require('http');
http.globalAgent.maxSockets = 100;
But it doesn't help. Please share your thoughts, thanks!
After leveraging on P.T.'s advice, the fix is:
app.get('/:version/query', function(req, res) {
res.setTimeout(4 * 60 * 60 * 1000); // nodejs socket timeout 4-hour
}
The express request 'close' events mean that the connection closed (I believe the express request inherits this from the underlying http.IncomingMessage.
This could be either because the client gave up, or because the nodejs socket timeout hit 2 minutes.
You need to figure out which side of the connection is giving up. I would've thought it was the client side, but was surprised to discover the nodejs timeout. See http://nodejs.org/api/http.html#http_request_setsocketkeepalive_enable_initialdelay for setting the socket timeout on the request's socket.
See also: node.js http.IncomingMessage does not fire 'close' event

ConnectJS + RequireJS hanging after a few reloads

I got a fairly massive requirejs based app that runs unbundled locally. I have a few hundred js files that get loaded in async. This is pretty quick locally and generally not a big deal. After maybe 10->20 page refreshes connectjs starts hanging for some reason. I got a half decent message once when I opened a different page and chrome indicated "waiting for available socket."
I'm guessing that at some point something ends up hanging and the connection never ends. At some point enough of these connections results in Node + connect to not accept any more requests. Has anyone experienced this and what is the solution? Is there a way to time out or reject requests from the server side?
Here is my connectjs server script:
var connect = require('connect');
var http = require('http');
var app = connect()
.use(connect['static'](__dirname))
.use(function (req, res) {
'use strict';
res.setHeader('Access-Control-Allow-Origin', '*');
// used to stub out ajax requests
if (req.url.indexOf('ajax/') !== -1) {
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({}));
}
});
var server = http.createServer(app);
server.listen(3000, function () {
'use strict';
console.log('server is listening on port 3000');
});

Resources