Using node.js with the npm firebase.
var firebase = require('firebase');
var blahFirebase = new firebase('https://myfirebase.firebaseIO.com/blah');
blahFirebase.once('value', function (snapshot) {
//
});
Why does node not exit when it is done reading the data?
In the new Firebase API you should use firebase.app.App.delete() to free the resources its holding. For example:
var app = firebase.initializeApp({ ... });
var db = firebase.database();
// Do something
app.delete(); // Release resources
Do not use process.exit() since it will stop the entire process (which is not what you would usually want).
My case is using firebase admin,
const admin = require('firebase-admin');
and I can end node process by
return admin.app().delete();
Update
Note that this is no longer applicable. Node.js will no longer hang when using once(), although it will be held open as long as there are active listeners subscribed to the remote server.
Original
The Firebase process opens sockets to the server and establishes listeners for incoming data on those connections. Just like a node web server, awaiting incoming HTTP connections, this holds the process open.
To end the process, you can simply utilize process.exit() from inside the callback:
blahFirebase.once('value', function (snapshot) {
//
process.exit();
});
setTimeout(()=> {
process.exit(1);
}, 1000);
Dunno why, but solved the problem.
Related
I googled a lot but still have no clear solution to my issue.
Connecting to MongoDB, usually you establish a connection and after the job is done you close it.
Since next.js (and probably node.js) is single threaded. Sometimes it happens that there are two requests processed async while one request established the connection to the database, the otherone is closing the exact same connection. So the first request runs into an Topology closed exception. I have the feeling that the mongodb driver client is shared.
Is there something I did not understood correct in this?
try {
await client.connect()
const database = client.db("test")
const collection = database.collection("test")
const newDataset = await collection.insertOne({})
return newDataset.insertedId.toString()
} finally {
await client.close();
}
As in the comments stated, ive seen a lot of examples & questions here on stackoverflow where in each received request (example below) a database connection is established. This has no benefits and is "bad" because it just takes time and makes no sense. E.g:
app.get("/", (req, res) => {
MongoClient.connect("...", (err, client) => {
// do what ever you want here
client.close();
});
});
If you application needs a database connection, establish the connection "in the startup phase" and keep the connection open. There is no reason to open and close the database connection for each request.
const mongodb = require("monogdb");
const express = require("express");
const app = express();
// some custom init stuff
// e.g. require your route handler etc.
mongodb.MongoClient("...", (err, client) => {
// do what ever you want with the db connection now
// e.g. monkey patch it, so you can use it in other files
// (There are better ways to handle that)
mongodb.client = client;
// or the better way
// pass it as function parameter
require("./routes")(app, client);
app.listen(8080, () => {
console.log("http server listening");
});
});
As you can see in the code above, we first create a database connection and then do other stuff. This has some advantages:
If your credentials are invalid, your application is not externeal reachable because the http server is not started
You have a single connection for all requests
Database queries are potential faster because you dont have to wait to establish first a db connection
NOTE: the code above was "inline coded" here and is not tested.
But i think its illustrated the concept behind my statement.
amqplib library lets you create a rabbitmq connection and that object will be a segue to doing other things such as creating a channel and etc.
suppose that I'm going for a Producer/Consumer pattern, where each time a user hits a specific route, a job is produced and sent to the rabbitmq server where it's processed by certain consumers(workers).
app.post("/routethatdelegatesheavywork", async (req,res) => {
const amqpServerLink =
"link-to-cloudmq";
const connection = await amqp.connect(amqpServerLink);
const channel = await connection.createChannel();
//do other stuff with channel
})
while this "works", but i don't want to re-create that connection every time the controller is invoked since it makes the producer very slow and it's really not how it's supposed to be done.
here is where my problem comes:
how do i initialize one connection and re-use it every time i need it?
i have tried to create a connection outside controllers and use it when necessary but it's not possible since the connection is promise-based and await doesn't work on entry point and it has to be inside an async function to work.
although it is possible to run await without async using ESM (es modules) i don't want to do so since i have written all of the application using CommonJS (require("package")), changing that would require me to go through a lot of files and change every import/export according to ESM.
So, is there any other way to create one connection(that is promise based) and re-use it without having to migrate to ESM syntax?
Yes, remember that require in nodejs are singletons. Make a new amqpServerInterface module, and do
const amqpServerLink = "link-to-cloudmq"
const connection = amqp.connect(amqpServerLink)
function connect() {
return connection
}
module.exports = {
connect
}
Then in your controllers
const amqpServerInterface = require('amqpServerInterface')
app.post("/routethatdelegatesheavywork", async (req,res) => {
const connection = await amqpServerInterface.connect();
const channel = await connection.createChannel();
//do other stuff with channel
})
This will always return the same connection promise and will resolve to the save connection.
There are a lot of examples of graceful stop for expressjs, how can I achieve the same for koajs?
I would like to disconnect database connections as well
I have a mongoose database connection, and 2 oracle db connection (https://github.com/oracle/node-oracledb)
I created an npm package http-graceful-shutdown (https://github.com/sebhildebrandt/http-graceful-shutdown) some time ago. This works perfectly with http, express and koa. As you want to add also your own cleanup stuff, I modified the package, so that you now can add your own cleanup function, that will be called on shutdown. So basically this package handles all http shutdown things plus calling your cleanup function (if provided in the options):
const koa = require('koa');
const gracefulShutdown = require('http-graceful-shutdown');
const app = new koa();
...
server = app.listen(...); // app can be an express OR koa app
...
// your personal cleanup function - this one takes one second to complete
function cleanup() {
return new Promise((resolve) => {
console.log('... in cleanup')
setTimeout(function() {
console.log('... cleanup finished');
resolve();
}, 1000)
});
}
// this enables the graceful shutdown with advanced options
gracefulShutdown(server,
{
signals: 'SIGINT SIGTERM',
timeout: 30000,
development: false,
onShutdown: cleanup,
finally: function() {
console.log('Server gracefulls shutted down.....')
}
}
);
I have answered a variation of "how to terminate a HTTP server" many times on different node.js support channels. Unfortunately, I couldn't recommend any of the existing libraries because they are lacking in one or another way. I have since put together a package that (I believe) is handling all the cases expected of graceful HTTP server termination.
https://github.com/gajus/http-terminator
The main benefit of http-terminator is that:
it does not monkey-patch Node.js API
it immediately destroys all sockets without an attached HTTP request
it allows graceful timeout to sockets with ongoing HTTP requests
it properly handles HTTPS connections
it informs connections using keep-alive that server is shutting down by setting a connection: close header
it does not terminate the Node.js process
Usage with Koa:
import Koa from 'koa';
import {
createHttpTerminator,
} from 'http-terminator';
const app = new Koa();
const server = app.listen();
const httpTerminator = createHttpTerminator({
server,
});
await httpTerminator.terminate();
To make sure the Oracle DB connections are closed nicely, you can use a connection pool and call pool.close() with a drainTime of 0 or greater. This will let the app relatively cleanly interrupt any operation that is currently using a connection. It allows freeing the DB end of the connections without the DB waiting for whatever timeout period to expire before it cleans itself up. Even with two connections this is a solution I'd look at, since it doesn't matter that the pool is small. You may need to set the Oracle Net out-of-band break detection as well, see Connections and High Availability.
Modern versions of node have support for AbortController, so no need for external libraries. A Simple example:
const app = new Koa();
const server = http.createServer(app.callback());
const controller = new AbortController();
server.listen({
host: 'localhost',
port: 80,
signal: controller.signal
});
// middleware... etc.
app.use(async (ctx) => {
ctx.body = 'Hello World';
});
// Later, when you want to close the server.
controller.abort();
I'm creating an Express server with reloadable endpoints.
To make this possible I created an endpoint for such a purpose.
But when I call server.close() on the Express's HTTP server it still continues listening, while the server.listening says otherwise, it still is.
Here is a simplified version of my script (Not working fully, but you get the gist):
class simpleServer {
constructor() {
let express = require('express');
this.app = express();
this.app.get('/reload', this.reload);
this.server = this.app.listen(3000);
}
reload(req, res) {
console.log('Closing server');
this.server.close(function() {
console.log('Closed server');
});
// Re-init & stuff
res.json({
message: 'Reloaded'
});
}
}
let server = new simpleServer();
When I call the endpoint, the server will output 'Closing server', but the 'Closed server' takes a long time to be called (5 minutes). And when I reload the page, it still works, while the server.listening is equal to false.
I'm using Node.js version 6.0.0 with Express version 4.14.0.
Some updates:
I fixed the issue by calling req.destroy() after sending the response, does this have any side-effects tho?
A cleaner fix would be keeping a record of current connections and closing those in the reload function instead of closing them instantly. This will probably be less heavy if you have a higher load.
When you call .close(), it only stops accepting new connections, it does not terminate existing connections.
The reason it may take some time to actually close is if there are existing connections that have set Connection: keep-alive in case of more requests.
You can use process.exit().
Or you can try
this.server.close(function() {
console.log('Closed server');
})();
I'm a noob to Node.js and Express :( I'm having an issue with accessing the socket.io object in other modules. I have created a global variable to hold a socket object however as soon as that user closes the connection the connection isn't available anymore. Technically I guess the connection still lives on but I delete it for resource reasons.
Notice below everytime a connection is made we assign that socket to the global variable which is accessible to all other modules.
// App.js
var io = require('socket.io').listen(server);
var sessionsConnections = {};
io.sockets.on('connection', function (socket)
{
global.socket = socket;
sessionsConnections[socket.id] = socket;
.....
}
socket.on("disconnect", function()
{
delete sessionsConnections[socket.id];
});
// Match.js
global.socket.emit('lobby:createMatch', data);
If the connection last assigned to the global closes Match.js will be screwed. At this point Match.js is the only module who will ever need that socket.io reference. Match.js has a bunch of exports for handling events, emitting the changes and rendering the view.
Are there any suggestions to how this is handled? Is it possible to instantiate an initial socket connection to live in App.js for the purpose of being a global reference?
The socket variable in io.sockets.on('connection', function(socket) {...}) is different for each connection.
Since in your code global.socket is always a reference to the socket relative to the last connected client, it is normal that if this client disconnects, this socket will die.
In any case, I don't see any reason to use a global variable. If you need to send a message to a specific client, you can use the socket variable inside the connection callback:
io.sockets.on('connection', function (socket)
{
socket.emit('foo', 'bar');
}
If you need to access the sockets in another module, you can export the sessionsConnections object, and access the socket you need by its id:
//In app.js
exports.connections = sessionsConnections;
//In match.js
var app = require('./app.js');
app.connections[clientId].emit('foo', 'bar');
Of course, you need to keep track of the id's somewhere.
You might try express.io,
http://express-io.org/
npm install express.io
It works the same as express, except that it has socket.io integration built-in. It also has basic routing, so that it is a little easier to deal with your issue.
Check out this simple example:
app = require('express.io')()
app.http().io()
app.io.route('some-event', function(req) {
req.io.broadcast('announce', 'someone triggered some-event')
})
app.listen(7076)
You can also do routing with objects, which makes it more like a traditional controller:
app = require('express.io')()
app.http().io()
app.io.route('posts', {
create: function(req) {
// create a post
},
remove: function(req) {
// remove a post
}
})
app.listen(7076)
Hope that helps!