With a single server setup, I receive events from the driver.
mongoose.connect('mongodb://localhost/mydb');
mongoose.connection.on('disconnected', function() {...});
mongoose.connection.on('error', function(err) {...});
When using a replica set (mongoose.connect('mongodb://localhost:27017/mydb,mongodb://localhost:27018/mydb');), shutting down all connected set members doesn't trigger those same events.
I'm not very familiar with the internals of the native driver and I'm wondering if this is a bug or if I need to manually detect this condition.
I'm using Mongoose 3.6.17 (mongodb driver 1.3.18)
Sans mongoose, I tried this with the same results (no events from a replica set).
require('mongodb').MongoClient.connect("mongodb://localhost:27017,localhost:27018/mydb", function(err, db) {
if (db) {
db.on('disconnected', function() {
console.log('disconnected');
}).on('error', function(err) {
console.log('error');
});
}
});
I've been having similar problems with Mongoose, asked on SO also. More recently, I've found this issue on Mongoose GitHub repository which led to this issue on the driver repository.
The Mongo driver wasn't emitting any of these events more than once, and today this has been fixed for single connections on v1.3.19.
It seems that it's a "won't fix" for now.
I ended up doing the following:
I set auto_reconnect=true
Until the application has connected to the database for the first time, i disconnect and reconnect. if i don't disconnect and reconnect, any queued queries won't run. after a connection has been established at least once, those queued queries do complete and then...
for single connections:
1. forked mongoose (to use mongodb to 1.3.19) so errors get triggered more than once.
2. catch the connection error and make the app aware of the disconnection, retrying until i give up and panic or the app is reconnected. how that's done is by pinging the server every x milliseconds with a command that will not queue:
var autoReconnect = mongoose.connection.db.serverConfig.isAutoReconnect;
mongoose.connection.db.serverConfig.isAutoReconnect = function(){return false;};
mongoose.connection.db.executeDbCommand({ping:1}, {failFast: true}, function(err) {
if (!err) {
// we've reconnected.
}
});
mongoose.connection.db.serverConfig.isAutoReconnect = autoReconnect;
for a replica set, i ended up polling the mongoose connection with the above ping every x milliseconds until i detect an error, in which case i set my local app state to disconnected and enter the reconnect poll loop above (2.).
here's a gist with the relevant bits. https://gist.github.com/jsas/6299412
This is a nasty inconsistency/oversight in mongoose.
Especially when developing a microservice where you're using a single server setup for development and replica set in production.
This is the way I ended up accurately tracking the status of my mongoose connection.
let alive = false;
function updateAlive() {
return mongoose.connection
&& mongoose.connection.readyState === mongoose.STATES.connected
// This is necessary because mongoose treats a dead replica set as still "connected".
&& mongoose.connection.db.topology.connections().length > 0;
}
mongoose.connection.on('connected', () => {
updateAlive();
// I think '.topology' is available on even single server connections.
// The events just won't be emitted.
mongoose.connection.db.topology.on('joined', () => updateAlive());
mongoose.connection.db.topology.on('left', () => updateAlive());
});
mongoose.connection.on('disconnected', () => {
updateAlive();
});
Related
I am using mongoose 4.13.7
I want to connect to mongo but if an error occurs I want to reconnect.
But after 5 reconnects, if an error occurs, the process should exit.
This is the code:
var count = 0;
handleDisconnect();
function handleDisconnect(){
count++;
console.log('Trying to connect to mongo. Attempt : ' + count);
mongoose.connect(config.mongo.uri,{useMongoClient:true});
mongoose.connection.on('error',(error)=>{
if (count >= 5){
console.log('Mongo ERROR');
console.error(error);
process.exit(1);
}
else{
setTimeout(handleDisconnect,1000);
}
});
mongoose.connection.on('open',()=>{
console.log('Connected to mongo at ' + Date.now());
});
}
I have posted the output of the code as well. I don't understand how the attempt count is exceeding 5? There is also a memory leak warning and node:6804 error message. What am I doing wrong?
Output of the code
You attach an event listener for the error event each time the event is emitted. This causes the memory leak warning and also makes the callback function run multiple times per each event ocurrance. You should not add the event handler inside the handleDisconnect() funciton.
Example:
function handleDisconnect(mongoError) {
// check error reason, increment counters, check if errors limit reached
}
mongoose.connection.on('error', handleDisconnect);
You need to check disconnected event when Mongoose lost connection to the MongoDB server. This event may be due to your code explicitly closing the connection, the database server crashing, or network connectivity issues.
mongoose.connection.on('disconnected', () => console.log('Server disconnected from mongoDB'));
error Event: Emitted if an error occurs on a connection, like a parseError due to malformed data or a payload larger than 16MB.
https://mongoosejs.com/docs/connections.html#connection-events
I have a mobile app using PouchDB that listens for changes from a remote CouchDB server. Occasionally the app pushes it's own changes up. Is there a way to check to see if I still have an active "connection" to the remote CouchDB instance?
The best solution I can come up with so far is to periodically call db.info and see if I get an error back, but that seems slightly risky.
As far as I could find out, PouchDB has no event to inform us that the syncing was stopped because of a disruption in the internet connection. I did a small hack to solve this.
When the sync is happening properly, and you make a change to your local PouchDB, the order in which events are fired is:
active
change
paused
However, when the internet is broken, the order is:
active
paused
There is no change event fired when syncing with the remote DB isn't taking place.
So, what I've done is made two flags for the active and change events , pouchDbSyncActiveEvent and pouchDbSyncChangeEvent respectively, and initialized both as false. Turn pouchDbSyncActiveEvent to true in the callback for the 'active' event. This will run regardless of whether there is internet or not. Turn pouchDbSyncChangeEvent to true in the callback for the 'change' event. If there is no internet, this won't run and pouchDbSyncChangeEvent will still remain false, and the 'paused' event will be fired soon after. This informs us that the syncing was stopped.
Code:
let pouchDbSyncActiveEvent = false
let pouchDbSyncChangeEvent = false
localPouchDb.sync(remoteCouchDb, {
live: true,
retry: true
})
.on('active', (change) => {
pouchDbSyncActiveEvent = true
})
.on('change', (change) => {
pouchDbSyncChangeEvent = true
})
.on('paused', (info) => {
if(pouchDbSyncActiveEvent == true && pouchDbSyncChangeEvent == false){
// Gotcha! Syncing with remote DB not happening!
}
else {
// Everything's ok. Syncing with remote DB happening normally.
}
pouchDbSyncActiveEvent = false
pouchDbSyncChangeEvent = false
})
Note:
You can write the .on event listener methods in the order you wish, but it is good to write them in the order they are executed :)
I'm assuming that you included the pouchdb tag because you're using PouchDB locally and are replicating between your local PouchDB and remote CouchDB. Assuming so, replication in PouchDB is an event emitter, so there are a bunch of different events you can define in your sync or replicate call, see the docs including the error event.
Did you try to look for an error in the paused event emitted when connection is lost?
localDB.replicate.to(remoteDB, {
live: true,
retry: true,
}).on('paused', (err) =>{
console.log('paused');
if (err) {
alert(`No connection! ${err}`);
}
// replication was paused, usually because of a lost connection
}).on('change', (change)=>{
// yo, something changed!
}).on('active', (info)=>{
// replication was resumed
}).on('error', (err)=>{
// totally unhandled error (shouldn't happen)
});
If you are using a live replication a paused event is emitted when the replication waits for a change and err is undefined. If there is a problem like a connection interruption a paused event is also emitted but with an err argument.
Note: I'm sure that there is safer ways to check if err is defined.
From the PouchDB documentation:
paused (err) - This event fires when the replication is paused, either
because a live replication is waiting for changes, or replication has
temporarily failed, with err, and is attempting to resume.
You need to set "retry" to false, or the error event won't get caught.
From the API reference:
error (err) - This event is fired when the replication is stopped due to an unrecoverable failure. If retry is false, this will also fire when the user goes offline or another network error occurs (so you can handle retries yourself, if you want).
Try this code:
// Sync pouch - couch
db.sync(couch, {
live: true,
retry: false
}).on('error', (err) => {
console.log(`A sync error occurred: ${err}`);
});
I'm having a weird issue with a TCP client - I use socket.connect() to connect to the server instance. However, since the server is not running, I receive an error of ECONNREFUSED (so far so good).
I handle it using on('error') and set a timeout to try and reconnect in 10 seconds. This should continue to fail as long as the server is down. which is the case.
However, as soon as the server is running, it looks like all of the previous sockets are still active, so now I have several client sockets connected to the server.
I tried to call the destroy at the beginning of the on('error') handler function.
Any ideas how to deal with that?
Thanks!
EDIT: Code snippet:
var mySocket;
var self = this;
...
var onError = function (error) {
mySocket.destroy(); // this does not change anything...
console.log(error);
// Wait 10 seconds and try to reconnect
setTimeout(function () {
console.log("reconnecting...");
self.once('InitDone', function () {
// do something
console.log("init is done")
});
self.init();
}, 10000);
}
Inside init function:
...
console.log("trying to connect");
mySocket = tls.connect(options, function () {
console.log("connected!");
self.emit('InitDone');
});
mySocket.setEncoding('utf8');
mySocket.on('error', onError);
...
The result of this is something like the following:
trying to connect
ECONNREFUSED
reconnecting...
trying to connect
ECONNREFUSED
reconnecting...
trying to connect
ECONNREFUSED
reconnecting...
--> Starting the server here
trying to connect
connected
init is done
connected
init is done
connected
init is done
connected
init is done
However I would expect only one connection since the previous sockets failed to connect. Hope this clarifies the question.
Thanks!
I have a Node.js application that contains an http(s) server.
In a specific case, I need to shutdown this server programmatically. What I am currently doing is calling its close() function, but this does not help, as it waits for any kept alive connections to finish first.
So, basically, this shutdowns the server, but only after a minimum wait time of 120 seconds. But I want the server to shutdown immediately - even if this means breaking up with currently handled requests.
What I can not do is a simple
process.exit();
as the server is only part of the application, and the rest of the application should remain running. What I am looking for is conceptually something such as server.destroy(); or something like that.
How could I achieve this?
PS: The keep-alive timeout for connections is usually required, hence it is not a viable option to decrease this time.
The trick is that you need to subscribe to the server's connection event which gives you the socket of the new connection. You need to remember this socket and later on, directly after having called server.close(), destroy that socket using socket.destroy().
Additionally, you need to listen to the socket's close event to remove it from the array if it leaves naturally because its keep-alive timeout does run out.
I have written a small sample application you can use to demonstrate this behavior:
// Create a new server on port 4000
var http = require('http');
var server = http.createServer(function (req, res) {
res.end('Hello world!');
}).listen(4000);
// Maintain a hash of all connected sockets
var sockets = {}, nextSocketId = 0;
server.on('connection', function (socket) {
// Add a newly connected socket
var socketId = nextSocketId++;
sockets[socketId] = socket;
console.log('socket', socketId, 'opened');
// Remove the socket when it closes
socket.on('close', function () {
console.log('socket', socketId, 'closed');
delete sockets[socketId];
});
// Extend socket lifetime for demo purposes
socket.setTimeout(4000);
});
// Count down from 10 seconds
(function countDown (counter) {
console.log(counter);
if (counter > 0)
return setTimeout(countDown, 1000, counter - 1);
// Close the server
server.close(function () { console.log('Server closed!'); });
// Destroy all open sockets
for (var socketId in sockets) {
console.log('socket', socketId, 'destroyed');
sockets[socketId].destroy();
}
})(10);
Basically, what it does is to start a new HTTP server, count from 10 to 0, and close the server after 10 seconds. If no connection has been established, the server shuts down immediately.
If a connection has been established and it is still open, it is destroyed.
If it had already died naturally, only a message is printed out at that point in time.
I found a way to do this without having to keep track of the connections or having to force them closed. I'm not sure how reliable it is across Node versions or if there are any negative consequences to this but it seems to work perfectly fine for what I'm doing. The trick is to emit the "close" event using setImmediate right after calling the close method. This works like so:
server.close(callback);
setImmediate(function(){server.emit('close')});
At least for me, this ends up freeing the port so that I can start a new HTTP(S) service by the time the callback is called (which is pretty much instantly). Existing connections stay open. I'm using this to automatically restart the HTTPS service after renewing a Let's Encrypt certificate.
If you need to keep the process alive after closing the server, then Golo Roden's solution is probably the best.
But if you're closing the server as part of a graceful shutdown of the process, you just need this:
var server = require('http').createServer(myFancyServerLogic);
server.on('connection', function (socket) {socket.unref();});
server.listen(80);
function myFancyServerLogic(req, res) {
req.connection.ref();
res.end('Hello World!', function () {
req.connection.unref();
});
}
Basically, the sockets that your server uses will only keep the process alive while they're actually serving a request. While they're just sitting there idly (because of a Keep-Alive connection), a call to server.close() will close the process, as long as there's nothing else keeping the process alive. If you need to do other things after the server closes, as part of your graceful shutdown, you can hook into process.on('beforeExit', callback) to finish your graceful shutdown procedures.
The https://github.com/isaacs/server-destroy library provides an easy way to destroy() a server with the behavior desired in the question (by tracking opened connections and destroying each of them on server destroy, as described in other answers).
As others have said, the solution is to keep track of all open sockets and close them manually. My node package killable can do this for you. An example (using express, but you can call use killable on any http.server instance):
var killable = require('killable');
var app = require('express')();
var server;
app.route('/', function (req, res, next) {
res.send('Server is going down NOW!');
server.kill(function () {
//the server is down when this is called. That won't take long.
});
});
var server = app.listen(8080);
killable(server);
Yet another nodejs package to perform a shutdown killing connections: http-shutdown, which seems reasonably maintained at the time of writing (Sept. 2016) and worked for me on NodeJS 6.x
From the documentation
Usage
There are currently two ways to use this library. The first is explicit wrapping of the Server object:
// Create the http server
var server = require('http').createServer(function(req, res) {
res.end('Good job!');
});
// Wrap the server object with additional functionality.
// This should be done immediately after server construction, or before you start listening.
// Additional functionailiy needs to be added for http server events to properly shutdown.
server = require('http-shutdown')(server);
// Listen on a port and start taking requests.
server.listen(3000);
// Sometime later... shutdown the server.
server.shutdown(function() {
console.log('Everything is cleanly shutdown.');
});
The second is implicitly adding prototype functionality to the Server object:
// .extend adds a .withShutdown prototype method to the Server object
require('http-shutdown').extend();
var server = require('http').createServer(function(req, res) {
res.end('God job!');
}).withShutdown(); // <-- Easy to chain. Returns the Server object
// Sometime later, shutdown the server.
server.shutdown(function() {
console.log('Everything is cleanly shutdown.');
});
My best guess would be to kill the connections manually (i.e. to forcibly close it's sockets).
Ideally, this should be done by digging into the server's internals and closing it's sockets by hand. Alternatively, one could run a shell-command that does the same (provided the server has proper privileges &c.)
I have answered a variation of "how to terminate a HTTP server" many times on different node.js support channels. Unfortunately, I couldn't recommend any of the existing libraries because they are lacking in one or another way. I have since put together a package that (I believe) is handling all the cases expected of graceful HTTP server termination.
https://github.com/gajus/http-terminator
The main benefit of http-terminator is that:
it does not monkey-patch Node.js API
it immediately destroys all sockets without an attached HTTP request
it allows graceful timeout to sockets with ongoing HTTP requests
it properly handles HTTPS connections
it informs connections using keep-alive that server is shutting down by setting a connection: close header
it does not terminate the Node.js process
Usage:
import http from 'http';
import {
createHttpTerminator,
} from 'http-terminator';
const server = http.createServer();
const httpTerminator = createHttpTerminator({
server,
});
await httpTerminator.terminate();
const Koa = require('koa')
const app = new Koa()
let keepAlive = true
app.use(async (ctx) => {
let url = ctx.request.url
// destroy socket
if (keepAlive === false) {
ctx.response.set('Connection', 'close')
}
switch (url) {
case '/restart':
ctx.body = 'success'
process.send('restart')
break;
default:
ctx.body = 'world-----' + Date.now()
}
})
const server = app.listen(9011)
process.on('message', (data, sendHandle) => {
if (data == 'stop') {
keepAlive = false
server.close();
}
})
process.exit(code); // code 0 for success and 1 for fail
I know socket.io has a built in feature for reconnecting and everything, however I don't think that it is working - as I have seen from others it's also not working for them either.
If a user puts their computer to sleep, it disconnects them, and then when they open it back up they are no longer connected so they don't any of the notifications or anything until they refresh the page. Perhaps it's just something that I'm not doing correctly?
var io = require('socket.io').listen(8080);
var users = {};
////////////////USER CONNECTED/////
console.log("Sever is now running");
io.sockets.on('connection', function (socket) {
//Tell the client that they are connected
socket.emit('connected');
//Once the users session is recieved
socket.on('session', function (session) {
//Add users to users variable
users[socket.id] = {userID:session, socketID:socket};
//When user disconnects
socket.on('disconnect', function () {
//socket.socket.connect();
var count= 0;
for(var key in users){
if(users[key].userID==session)++count;
if(count== 2) break;
}
if(count== 1){
socket.broadcast.emit('disconnect', { data : session});
}
//Remove users session id from users variable
delete users[socket.id];
});
socket.on('error', function (err) {
//socket.socket.connect();
});
socket.emit("connection") needs to be called when the user reconnects, or at least the events that happen in that event need to be called.
Also socket.socket.connect(); doesn't work, it returns with an error and it shuts the socket server down with an error of "connect doesn't exist".
The problem is related to io.connect params.
Look at this client code (it will try to reconnect forever, with max delay between attempts 3sec):
ioParams = {'reconnection limit': 3000, 'max reconnection attempts': Number.MAX_VALUE, 'connect timeout':7000}
socketAddress = null
socket = io.connect(socketAddress, ioParams)
There are two important parameters out there, related to your problem:
reconnection limit - limit the upper time of delay between reconnect attemts. Normally it's getting bigger and bigger in time of server outage
max reconnection attempts - how many times you want to try. Default is 10. In most cases this is the problem why the client stops trying.