Why is the MongoDB Node Driver generating instance pool destroyed errors? - node.js

When I run the following code I am getting the error message 'MongoError: server instance pool was destroyed'. Any idea why or how to fix this?
var csv = require('./importer.js');
var MongoClient = require('mongodb').MongoClient;
var assert = require('assert');
var ObjectId = require('mongodb').ObjectID;
var url = 'mongodb://.....';
MongoClient.connect(url, function(err, db) {
assert.equal(null, err);
console.log("Connected correctly to server.");
csv.foreach('data/airports.csv', function(airport){
db.collection('airports').insertOne(airport, function(err, result) {
if(err) {
console.log(err)
} else {
console.log("Inserted: " + airport.ident);
}
});
});
db.close();
});

csv.foreach and the insertOne calls are (presumably) both async, so you're calling db.close() before your inserts have completed.
You need to come up with a way of waiting to call db.close() until all your inserts' callbacks have been called. How to do that depends on how your csv module works, but using something like the async module can help with the async flow control.

I have same issue, after calling 'db.close()' together with async npm, this problem is resolved.

Try using a for...of instead of forEach. This solved my problem, although I am using async/await.

Related

Unable to connect to MongoDB from NodeJS, deployed as action in OpenWhisk

I have the following code written a file called app.js. MongoDB is installed on 192.168.16.1, which is my laptop. When I run this using node app.js command, I get a message "connected".
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var mongoose = require('mongoose');
var MongoClient = require('mongodb').MongoClient;
// Connect to the db
MongoClient.connect("mongodb://192.168.16.1:27017/angularcrud", function (err, db) {
if(err) {console.log(err); }
else {console.log('connected');}
});
\
I have an OpenWhisk environment setup on my laptop using Vagrant. If is ssh to vagrant and ping to 192.168.16.1, I get ping response, so I am sure that vagrant VM is able to reach 192.168.16.1. I have written the following code in NodeJS to create an OpenWhisk action. I have deployed it into openwhisk as a .zip file (which includes Node_modules folders also).
function entryPoint(args) {
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var message = "Connection not SET";
var mongoose = require('mongoose');
var MongoClient = require('mongodb').MongoClient;
// Connect to the db
MongoClient.connect("mongodb://193.168.16.1:27017/angularcrud", function (err, db) {
if(err) {return err;}
else {return 'success';}
});
}
module.exports.main = entryPoint;
If I run the above code in OpenWhisk, I get a result {}. If I remove the MongoClient.Connect statement and return a simple string, then I am getting the string when I invoke the action. I am sure there is something wrong in the MongoClient.Connect, when run on OpenWhisk. But, I am really stuck, because I get no error to tell me what is going wrong.
The entryPoint function executes an asynchronous function to connect to the database. When executing asynchronous function calls, you need to return a Promise from the action handler. This ensures the platform will block on that asynchronous result before completing the invocation.
function main() {
return new Promise((resolve, reject) => {
MongoClient.connect(URL, (err, db) => {
if(err) return reject(err)
resolve({message: "success"})
})
})
}

How to access the value of variable outside the function in node js (node js with postgresql)

var pg = require('pg');
var conString = "postgres://postgres:abk#localhost/bot";
var res = [];
pg.connect(conString, function(err, client, done) {
if(err) {
return console.error('error fetching client', err);
}
client.query('SELECT * FROM bot.questions', function(err, result)
{
done();
if(err) {
return console.error('error running query', err);
}
res.push(result.rows[0]);
console.log(res); //inside function scope
});
});
console.log(res); // outside function scope
"console.log()" which has called inside the function gives proper resulting array, but outside function the same array variable shows an empty array. I have tried lot of things, such as callbacks, promises functionalities in node js but was not able to see the resulting "res" outside the function.
Please suggest me how do I make that "res" variable accessible from outside of function.
NOTE: Specifically, I need "console.log(res)" to print the "res" outside of function as mentioned in above code.
res variable is indeed accessible outside the function, as it is defined outside the function
res variable is indeed populated outside of the function, as it is seen as populated inside the function
root cause of your problem is that the outside print happens chronologically earlier than the inside activity, as the function is asynchronous, the call comes out immediately, and proceeds to execute the last line in the code.
If you are just particular about getting the value outside, it is indeed is the case
If you are wanting to just print the value outside, use a delayed callback in which to print the content
If you are wanting to post-process the value outside, you will need to link the connect function with a next function, through synchronization primitives.
Hope this helps.
As explained before, the query's result will exist only when the callback is fired, so it won't make sense accessing it in a part of the code that would execute before that.
It seems like you need to read a bit about node's asynchronous programming.
I suggest you read about promises and generators.
I found a solution finally, I have created seperate file for db config along with resulting array of values using callback function to export. Here is the "db.js" file which I have created.
var pg = require('pg');
var conString = "postgres://postgres:abk#localhost/chatbot";
var resp = [];
function executeQuery (callback) {
pg.connect(conString, function(err, client, done) {
if(err) {
return console.error('error fetching client', err);
}
client.query('SELECT * FROM bot.questions', function(err, result) {
done();
if(err) {
return console.error('error running query', err);
}
resp.push(result.rows[0]);
// console.log(res);
callback(resp);
});
});
};
exports.executeQuery = executeQuery;
and this I have imported into my "app.js" like bellow
var express = require('express')
var bodyParser = require('body-parser')
var request = require('request')
var app = express()
var db = require('./db')
app.post('/webhook/', function (req, res) {
db.executeQuery(function ( resp) {
console.log(resp);
});
});
This is how I can now access "resp" as resulting array, and can pass it to any function as an argument wherever required.
There is specific need as per my requirements to make the db config file separate from app.js file, otherwise I know it can also be manipulated in same app.js file.
So finally, This works for me perfectly.
This is what a kind of example I was expecting. But I found it by myself.
Anyways thanks for your involvement, and suggessions.

mongoose .save() doesn't work

I have those code running try to save an object into MongoDB.
The .save() never got success run. the code running fine.
.save() method doesn't work.
var conn = mongoose.createConnection(mongoUrl, {auth: {authdb: "admin"}});
conn.on('error', function (err) {
throw err;
});
conn.once('open', function callback() {
console.log("connected to " + mongoUrl);
var cacheSchema = mongoose.Schema({}, {strict: false});
cacheSchema.set('collection', 'caches');
// you need to specify which connection is uing.
var Cache = conn.model('cache', cacheSchema);
var measure = new Cache();
measure['test'] = "test";
measure.save(function(err){
console.log('test');
});
});
I just ran into a similar issue in my code. For mine, I was dealing with an object within my user document. I had to run a user.markModified('object') before the user.save() to ensure the changes were saved to the database.
My running theory is that Mongoose wasn't tracking items unset or removed from the database automatically
Please read this part of documentation from mongoose and try the following:
var measure = new Cache({test:'teste'});
// or measure.set('test', 'teste');
measure.save(function (err) {
console.log(err);
});
You will be able to see the issue if there's any.
Update the issue is using:
var Cache = conn.model('cache', cacheSchema);
instead of
var Cache = mongoose.model('cache', cacheSchema);

Connect synchronously to mongodb

I would like to connect to mongodb first, then run everything else in my application.
To do it I have to write something like:
MongoClient.connect("mongodb://localhost/test", function(err, connection) {
if (err) { console.error(err); }
db = connection;
var app = express();
// Include API V1
require("./apiv1.js")(app, db);
app.listen(3000, function(err) {
if (err) { console.error(err); } else { console.log("Started on *:3000"); }
});
});
This makes my app to be completely indented inside the .connect function... Which looks ugly and takes space while I work on my project.
I think the best solution would be have the MongoDB connection synchronous (even because witout the DB connection my app cannot work so why should I do something while it's connecting?) and then run the rest of my code.
How can I do?
You can't connect to MongoDB synchronously, but you may get rid of this ugly callback from your code.
The best way to do it is to adopt some wrapper around node-mongodb-native driver.
Take a look at the following modules.
mongojs
var mongojs = require('mongojs');
var db = mongojs('localhost/test');
var mycollection = db.collection('mycollection');
mongoskin
var mongo = require('mongoskin');
var db = mongo.db("mongodb://localhost:27017/test", {native_parser:true});
monk
var monk = require('monk');
var db = monk('localhost/test');
var users = db.get('users')
Of course, internally all of them are establishing MongoDB connection asynchronously.
Using the async library, you can aleve some of these issues.
For example in my server startup I do the following :
async.series([
function(callback){
// Initialize the mongodb connection and callback on completion in init.
db.init(function(){
callback();
});
},
function(callback){
// Listen on requests etc.
webServer.init(function(){
callback();
});
},
function(callback){
// Set up anything else that I need
callback();
}
]);
If you are using Node 6 and up versions, you can do something like this:
const MongoClient = require('mongodb').MongoClient;
const url = 'mongodb://localhost:27017/mydb';
let db = null;
getdb();
//your code
async function getdb() {
db = await MongoClient.connect(url);
}
Bring the mongodb library.
Declare the url constant .
Declare the variable db as null.
Call the getdb function.
Create the getdb function which has firt the async word
Assign to the db variable the result of the connection with the key word await.
You can do it with thunky, thunky executes an async function once and caches it, the subsequent calls are returned from the cache.
const MongoClient = require('mongodb').MongoClient;
const thunky = require('thunky');
var connect = thunky(function(cb){
let url = 'mongodb://localhost:27017/test';
MongoClient.connect(url, function(err, client){
console.log('connecting')
cb(err, client);
})
})
connect( (err, client) => {
console.log('connection 1')
})
connect( (err, client) => {
console.log('connection 2')
})
connect( (err, client) => {
console.log('connection 3')
console.log('closing')
client.close();
})
*Note: I am using latest 3.x mongodb driver

mongodb native how to work with query

I'm trying to retrieve data from my mongolab DB using Node-mongodb-native
var findAll = function () {
var ddocs;
collection.find({}).each(function (arr, docs) {
ddocs = docs;
});
console.log(ddocs);
};
But it seems that when I log ddocs, it will give me undefined, but if I log docs it will show me the data.
Please help
How should I use this function ?
Thanks
Tzelon Machluf
You're basically trying to create a function that will return all of the documents in a collection? If so, the below should do work. However, I agree with #hgoebl and that you should work on your understanding of node as this is not likely the best way to accomplish what you're trying to do.
var ddocs;
var findAll = collection.find().toArray( function(err, docs) {
if(err)
throw err;
console.log('Collection returned');
return ddocs = docs;
});
setTimeout( function(err) {
if(err) throw err;
console.log(ddocs);
db.close();
}, 1000);
One thing in particular to note: collection.find is asynchronous, so the problem in your code (why it says ddocs is undefined) is that you're logging ddocs before the collection.find ever finished; thus, it's initialized, but no assigned any values. The Node convention is to nest callbacks so that everything happens in the right order and when it's ready.

Resources