How mongodb connection works on concurrent requests in NodeJS express server? - node.js

I am new to mongoDB and I'm currently working on setting it up with Node express server. I wonder how to manage concurrent requests to the mongodb to read the collection data using the mongoose driver module.
For example:
If 100 users are accessing my server at a time (http://xxxxxx.com/showusers), how will the mongodb connection in the express server work? Will it be a single connection or split into 100 connections, one for each request?
How can I close the connection object in mongodb efficiently after the operation? Or can we leave the connection in the express server as in the below code?
Here follows my code..
Server.js
var express = require('express');
var app = express();
app.set('port', config.port);
app.get('/users',storeusersapi.showUsers);
app.get('/storeUser',storeusersapi._insertUserDetails);
app.get('/findUser/:email',storeusersapi._findUser);
app.listen(app.get('port'),function(){
log.info('Express app started on port ' + app.get('port'));
});
storeusersapi.js
var mongoose = require('mongoose');
var log = require('../config/logger');
// Mongoose connection to MongoDB (ted/ted is readonly)
mongoose.connect('mongodb://localhost/mydb', function (error) {
if (error) {
log.error("MongoDB Connection failure - " +error);
}else{
log.info('MongoDB is connected Successfully!!!');
}
});
// Mongoose Schema definition
var Schema = mongoose.Schema;
var UserSchema = new Schema({
first_name: String,
last_name: String,
email: String
});
// Mongoose Model definition
var User = mongoose.model('users', UserSchema);
exports.showUsers = function(req,res){
User.find({}, function (err, docs) {
res.json(docs);
});
};
exports._insertUserDetails = function(req,res){
var object = new User({first_name:'bob',last_name:'sel',email:'sel#xxxxx.com'});
object.save(function (err) {
if (err) {
log.error('Insertion error - '+ err);
}
else {
log.info("User Stored into database!!!");
}
});
};
exports._findUser = function(req,res){
User.find({ email: req.params.email }, function (err, docs) {
res.json(docs);
});
};

I have answered for both of your question separately.
1. How will the mongodb connection in the express server work?
Once a connection is created to the mongodb database.(using mongoose or any other framework) It will create a pool of connections with that. (Mongoose default pool size is 5, 100 in python) The created connection pool is maintained by the driver therefore those connections can be re-used when connections to the database are required.
The best practice is to create a new connection only once for the whole application. Once connection is created the connection object will be used as a singleton. When you connect to the database using mongoose models, separate connections are allocated from the created connection pool.
If you are going to create a new connection each time then It will cause to a connection churn.
2. How can I close the connection object in mongodb efficiently after the operation ?
I am not sure 100% about this answer. My suggestion is to disconnect the connection when the express application exits.
var db = mongoose.connect('mongodb://localhost:27017/database');
db.disconnect();
According to my knowledge what you have don in the code is correct. You have created a new connection only once. Since the connection pool is created with that you don't need to create more connections.
Please go through this link to get a clear understanding on connection pools and their usage.
https://dzone.com/articles/deep-dive-connection-pooling

Related

Mongodb connection number increase [duplicate]

I am making a single connection to MongoDB via Mongoose in Node.js Express app:
var express = require('express');
var mongoose = require('mongoose');
mongoose.connect('localhost', 'test');
I then define schema, followed by models, and lastly the controller that pulls all users from the database:
app.get('/users', function (req, res) {
return User.find(function (err, users) {
if (!err) {
return res.send(users);
}
});
});
These DB connections open during the application start and they stay open for the duration of the node.js application.
What bothers me is why do I have 5 connections open? As soon as I close the node.js app, all 5 connections are closed.
Related note: For a REST API server is it better to have MongoDB connection always open. Or is it better to manually open/close connections per each user request?
d
That's because Mongoose uses a pool of 5 connections (by default) that are shared throughout your application. For best performance, it's best to just leave them open.
You can alter the default behavior via the options parameter to mongoose.connect. For example:
mongoose.connect('localhost', 'test', { server: { poolSize: 3 }}); // Use 3 connections
Add this if statement to check only when mongoose disconnected, try to connect it
if (Mongoose.connection.readyState === 0)
mongoose.connect('localhost', 'test');
readyState contain these types :
0: disconnected
1: connected
2: connecting
3: disconnecting

NodeJS DB2 Connection pooling for Express app

I see very few online posts when it comes to NodeJS with IBM DB2. I am new to NodeJS and having issues to configure connection pooling for my web app. I am successfully running node app with single connection in my local but not sure how to configure connection pooling. Below code is how I have it for single connection.
DBConnection JS:
module.exports = function(dbConnection)
{
var Pool = require('ibm_db').Pool;
var pool = new Pool();
pool.open("MY_CONNECTION_STRING",function(err,connection){
//error handling logic ...
dbConnection(connection);
});
}
App listener js:
var express = require('express');
var app = express();
app.listen(8080,function(){
console.log("server started..);
});
require('./DBConnection')(function(connection){
app.get('/getEmpId',function(req,res){
connection.query("MY_SQL_QUERY",function(error,results,fields){
//some other logic
res.send(results[0]);
//Closing connection
connection.close(function(err2) {
if(err2) console.log(err2);
});
});
});
}
Need your suggestions to setup connection pool where I can use one connection for each request when concurrent users are accessing and close the connection after serving request.
You can take a look at the brief samples provided with the IBM node-ibm_db driver. It has a section on Connection Pooling. The driver reuses the node-odbc pool and you need to invoke the open/close calls on the Pool object. Here is the sample taken from the node-ibm_db site:
var Pool = require("ibm_db").Pool
, pool = new Pool()
, cn = "DATABASE=dbname;HOSTNAME=hostname;PORT=port;PROTOCOL=TCPIP;UID=dbuser;PWD=xxx";
pool.open(cn, function (err, db) {
if (err) {
return console.log(err);
}
//db is now an open database connection and can be used like normal
//if we run some queries with db.query(...) and then call db.close();
//a connection to `cn` will be re-opened silently behind the scense
//and will be ready the next time we do `pool.open(cn)`
});

Mongoose never returns its promise when I try to connect to the same db using a working URL

I have a locally hosted mongodb that I can connect to using mongodb.MongoClient.
Working code:
var mongoClient = require("mongodb").MongoClient;
...
var startApp = function(db) {
// Get our collections in an easy to use format
var database = {
chats: db.collection('chats'),
messages: db.collection('messages')
};
// Configure our routes
require('./app/routes')(app, database);
// START APP
// Start app on port
app.listen(port);
// Tell user the app is running
console.log("App running on port " + port);
// Expose app
exports = module.exports = app;
}
// DATABASE
var database = null;
mongoClient.connect(config.url, function(err, returnDB) {
if(err) {
console.log(err);
} else {
console.log("DB connected");
startApp(returnDB);
}
});
Legacy code that no longer works:
var mongoose = require('mongoose');
...
// Connect to DB
console.log('Connect to database (' + db.url + ')');
mongoose.connect(db.url);
I have added a callback to this connect method but it never gets called (error or no error, this connect function never gets to my callback).
This entire legacy app relies on the API using mongoose to talk to the database so I do not want to redo it all using mongodb. How can I fix this?
*config.url and db.url are loaded from the same file and it is a valid and running mongodb.
It was really easy to fix. Thanks #Bhavik for asking me what version I was using.
I updated mongoose to 4.8.1 by specifying the newest version in packages.json and the issue is resolved.

mongoose.createconnection missing database error

What I am trying to do is getting list of databases on my localhost server I get the ip address which is in this case local host and port of database server and calling the service Below.
The error I am getting in console is missing database name as I have to connect only database server where am I missing some thing?
app.post('/loadDataBase', function(req,res){
app.set('mongoose').connection.close();
var mongoose = require('mongoose')
, Admin = mongoose.mongo.Admin;
// create a connection to the DB
var connectionStr="mongodb://"+req.body.host+":"+req.body.port;
/* var connection = mongoose.createConnection(
'mongodb://127.0.0.1');*/
var connection = mongoose.createConnection(connectionStr) ;
connection.on('open', function() {
// connection established
new Admin(connection.db).listDatabases(function(err, result) {
// database list stored in result.databases
var allDatabases = result.databases;
res.send(allDatabases);
});
});
});'
app.set('mongoose').connection.close(); Will this close my previous mongoose connection?
This works for me with a mongod running locally on port 27017. It successfully prints out a list of databases, along with their sizeOnDisk.
var mongoose = require('mongoose')
, Admin = mongoose.mongo.Admin;
var connection = mongoose.createConnection('mongodb://localhost:27017') ;
connection.on('open', function() {
// connection established
new Admin(connection.db).listDatabases(function(err, result) {
// database list stored in result.databases
var allDatabases = result.databases;
console.log(allDatabases);
});
});
Can you give me your exact error output? This maybe could help me diagnose what the problem is.
As for your question about closing a previous mongoose connection: you can close the connection with mongoose.disconnect(). However, instead of connecting and disconnecting repeatedly, it is better to connect once when your application starts up and then disconnect when your application shuts down.

Creating a collection schema in a mongolab mongodb database from node.js

I'm new to node.js and mongodb.
I'm trying to create a schema for a User collection in a mongolab mongodb database from a node.js app with the code below. The code does not seem to be failing (at least, I get no error messages), but I don't see any indication that it is succeeding either. That is, when I go to mongolab and look at my database, I don't see that any schema was created - https://dzwonsemrish7.cloudfront.net/items/01263Y1c312s233V0R17/mongodb-schema.png?v=7fdc20e3.
Can someone explain what I might be doing wrong, or how I can verify that my code succeeded and a schema was, in fact, created for my collection?
// file: app.js
var express = require('express'),
http = require('http'),
mongoose = require('mongoose');
var app = express(),
port = 3000;
// Connect to database in the cloud (mongolab)
mongoose.connect('mongodb://username:password#ds041344.mongolab.com:41344/stockmarket');
// Create a schema for User collection
mongoose.connection.on('open', function () {
console.log(">>> Connected!");
var UserSchema = new mongoose.Schema({
username: {type: String, unique: true},
password: String
});
var UserModel = mongoose.model('User', UserSchema);
});
app.get('/', function(req, res){
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello, World!\n');
});
http.createServer(app).listen(port, function(){
console.log("Express server listening on port " + port + " ...");
});
You must insert a document first. Schemas are not explicitly defined in mongodb. Once you insert a document, the collection will automatically be created and you will see it in the mongolab console.
Example from http://mongoosejs.com/
var mongoose = require('mongoose');
var db = mongoose.createConnection('localhost', 'test');
var schema = mongoose.Schema({ name: 'string' });
var Cat = db.model('Cat', schema);
var kitty = new Cat({ name: 'Zildjian' });
kitty.save(function (err) {
if (err) // ...
console.log('meow');
});
after the save call above the collection will be created
Data in MongoDB has a flexible schema. Documents in the same collection do not need to have the same set of fields or structure, and common fields in a collection’s documents may hold different types of data.

Resources