NodeJS + MongoJS: Nested Callbacks Issue - node.js

I'm still a n00b with NodeJS so this question may be somewhat elementary.
I'm using MongoJS to read two logically related collections. The first find() returns a value that I pass to the second find() to get the information I need.
I've tried several strategies, the last one (snippet #1) being a class that I export.
Before that I just had a function that did a return, returning the desired value, i.e., "config[0]".
In this code all I did was set the "sapConfig" attribute to the word "test", but when I execute this code the value of "sapConfig" is always "null" after I call the "get_config()" method and - strangest of all - the reference to "this.sapConfig = 'test'" generates an error, i.e., "Cannot set property 'sapConfig' of undefined".
When I had the code just as a simple function with a return statement (snippet #2), no errors were generated but the value returned is always "undefined" although the console.log() statements show that the value of the variable being returned has the desired value. What gives?
Code Snippet #1: Returns Object
"use strict";
var mongojs = require('mongojs'); // MongoDB API wrapper
module.exports = function(regKey) {
this.regKey = regKey;
this.sapConfig = null;
this.get_config = function() {
// Read SAP connection information from our MONGO db
var db = mongojs('mongodb://localhost/MIM', ['Configurations','Registrations']);
db.Registrations.find({ key: this.regKey }, function(err1, registration){
console.log('Reg.find()');
console.log(registration[0]);
db.Configurations.find({ type: registration[0].type }, function(err2, config){
console.log('Config.find()');
console.log('config=' + config[0].user);
this.sapConfig = 'test';
});
});
}
this.get_result = function() {
return this.sapConfig;
}
}
Again, the code in snippet #1, when I make a call to "get_config()", results in an error when it executes the line "this.sapConfig = 'test'".
However, after this error I can execute "obj.get_result()" and I get the value to which it was initialized, i.e., null. In other words, that same code doesn't generate an error saying that the "this" is undefined as .in the "get_config()" method
Code Snippet #2: Using the "return" statement
"use strict";
var mongojs = require('mongojs'); // MongoDB API wrapper
module.exports = function(regKey) {
// Read SAP connection information from our MONGO db
var db = mongojs('mongodb://localhost/MIM', ['Configurations','Registrations']);
db.Registrations.find({ key: regKey }, function(err1, registration){
console.log('Reg.find()');
console.log(registration[0]);
db.Configurations.find({ type: registration[0].type }, function(err2, config){
console.log('Config.find()');
console.log('config=' + config[0].user);
return config[0].user;
});
});
}
When I receive the return value and inspect it, it's "undefined". For example, at the Node CL I issue the following commands:
var config = require('./config') // The name of the module above
> var k = config('2eac44bc-232d-4667-bd24-18e71879f18c')
undefined <-- this is from MongoJS; it's fine
> Reg.find() <-- debug statement in my function
{ _id: 589e2bf64b0e89f233da8fbb,
key: '2eac44bc-232d-4667-bd24-18e71879f18c',
type: 'TEST' }
Config.find()
config=MST0025
> k <-- this should have the value of "config[0]"
undefined
You can see that the queries were successful but the value of "k" is "undefined". What's going on here?
I don't care which approach I use I just need one of them to work.
Thanks in advance!

this.sapConfig is not accessible. Thats because this refers to within the current function. What I like todo, is have a variable that refers to the function instance that you know sapConfig is located in.
Ex:
function Foo() {
var self = this;
this.test = "I am test";
var bar = function(){
return function(){
console.log(this.test); //outputs undefined (because this refers to the current function scope)
console.log(self.test); //outputs "I am test";
}
}
}
Here is your first code snippit with my example implemented:
"use strict";
var mongojs = require('mongojs'); // MongoDB API wrapper
module.exports = function(regKey) {
var self = this;
this.regKey = regKey;
this.sapConfig = null;
this.get_config = function() {
// Read SAP connection information from our MONGO db
var db = mongojs('mongodb://localhost/MIM', ['Configurations', 'Registrations']);
db.Registrations.find({ key: this.regKey }, function(err1, registration) {
console.log('Reg.find()');
console.log(registration[0]);
db.Configurations.find({ type: registration[0].type }, function(err2, config) {
console.log('Config.find()');
console.log('config=' + config[0].user);
self.sapConfig = 'test';
});
});
}
this.get_result = function() {
return self.sapConfig;
}
}
For your second snippet. You are trying to return a value from within your nested callback. Since nested functions are asyncronous, you cannot do that.
Here is how I like to return values from nested callbacks:
Ex2:
//Function example
var functionWithNested = function(done) {
//Notice the done param.
// It is going to be a function that takes the finished data once all our nested functions are done.
function() {
//Do things
function() {
//do more things
done("resultHere"); //finished. pass back the result.
}();//end of 2nd nested function
}(); //end of 1st nested function
};
//Calling the function
functionWithNested(function(result) {
//Callback
console.log(result); //resultHere
})
Here is your code using that example:
"use strict";
var mongojs = require('mongojs'); // MongoDB API wrapper
module.exports = function(regKey, done) {
// Read SAP connection information from our MONGO db
var db = mongojs('mongodb://localhost/MIM', ['Configurations', 'Registrations']);
db.Registrations.find({ key: regKey }, function(err1, registration) {
console.log('Reg.find()');
console.log(registration[0]);
db.Configurations.find({ type: registration[0].type }, function(err2, config) {
console.log('Config.find()');
console.log('config=' + config[0].user);
done(config[0].user);
});
});
}
//Then wherever you call the above function use this format
// if config is the name of the function export above...
new Config().(regKey, function(result){
console.log(result); //config[0].user value
})
Lots and lots of code, but I hope you were able to follow it. Let me know if you have any more questions! Cheers.

Related

Node.js module.exports parent/child variable reference

In node.js I have this scenario:
main.js
module.exports = {
dbHandler: {}
}
const DB_CONNECT = require('dbConnect.js');
const CHILD_MODULE = require('childModule.js');
module.exports.dbHandler = DB_CONNECT.connectDB(); // establishes the connection to the sqlite3 db
// ... give some time to module.exports.dbHandler to be loaded. (lab testing)
CHILD_MODULE.queryDB(); // <----- error occurs
childModule.js
var db = module.parent.exports.dbHandler;
//issue is here. Even after the parent have set dbHandler, this still empty {}.
module.exports.queryDB = function(){
db.all('SELECT * from mytable', (err, rows) => { // callback
console.log(rows);
}
Since DB_CONNECT.connectDB() is async, I give it a while (lab test) to load the database and updating module.exports.dbHandler before calling CHILD_MODULE.queryDB()
the error occurs when db.all is called.
TypeError: db.all is not a function
db still an empty object {}.
What is wrong in this code? How do I make the child's db to access the parent's module.exports.dbHandler ?
First of all, I will not fix your problem directly. I will try to explain my comment in above.
I have had a similar scenario in one of my projects. But I have used MongoDB. My db model looks like this:
var MongoClient = require('mongodb').MongoClient
var url = process.env.MONGO_URI
var collection = 'shortlinks'
var state = {
db: null
}
exports.connect = function (done) {
if (state.db) return done()
MongoClient.connect(url, function (err, db) {
if (err) return done(err)
state.db = db
done()
})
}
exports.get = function () {
return state.db
}
...
and some other methods
And I have accessed this module from different places for the same database connection with this line:
var db = require('../models/db')
I can access the same db instance with getter method and other methods as well.

Use generator function with yield inside a class in ES2015 using KoaJS

I’m trying to call a generator function inside a class from the constructor, it runs but nothing happens (my console.log are not printing) as if the generator function is never called.
Update #1:
Here's an updated version of my code. I am able to access my findOne function with the next() function, but since I wrapped "users" with co-monk, I tough there was no need to call it. I'm still not sure why I need to call next(0) function 2 times to jump over the yield call.
Therefore, I'm now getting undefined when I print the output of "userData".
If my issue is related to understand on how yield works, maybe pointing me to direction could help me. I tried using generator functions with yield calls apart from a class and it worked perfectly fine with monk/co-monk.
Update #2:
I should also mention that I'm using babel6 for transcript.
"use strict";
var monk = require("monk");
var wrap = require("co-monk");
var db = monk("localhost/test");
var users = wrap(db.get("users"));
class User {
constructor(user) {
if (typeof user == "object") {
var findUser = this.findOne(user.id);
findUser.next();
findUser.next();
if(findUser != null){
this._user = user;
}
}
else {
console.error("user parameter is not an oject");
return false;
}
}
*findOne(id) {
var userData = yield users.findOne({_id: id});
console.log(userData); // Getting undefined
this._user = userData;
};
}
var _user = new User({id : "1234"});
console.log(_user);
export default User;
It looks like the docs for co-monk are a bit vague, but looking at the one test it has, it needs to be used within co.
var co = require("co");
// rest of your code
findOne(id) {
return co(function* () {
var userData = yield users.findOne({_id: id});
console.log(userData); // Getting undefined
this._user = userData;
});
};
What it won't allow you to do is seamlessly transition async code back into sync code, so this.findOne() will return a promise. As a result, your User object may not be populated immediately after called new User({id : "1234"}).
tried with the lib co and it worked.
It looks like co-monk is broken inside a generator function inside a class at this moment.
so here's what worked for me :
"use strict";
var monk = require("monk");
//var wrap = require("co-monk");
var db = monk("localhost/test");
var users = (db.get("users"));
var co = require('co');
class User {
constructor(user) {
if (typeof user == "object") {
var findUser = this.findOne(user.id);
this._user = user;
}
else {
console.error("The user params is not an object");
}
}
findOne(id) {
co(function* () {
return yield users.findOne({_id: id});
}).then(function (val) {
console.log(val);
}, function (err) {
console.error(err.stack);
});
};
}

How to return Node.js callback

I have a node.js method that using mongoose in order to return some data, the problem is that since I'm using a callback inside my method nothing is being returned to the client
my code is:
var getApps = function(searchParam){
var appsInCategory = Model.find({ categories: searchParam});
appsInCategory.exec(function (err, apps) {
return apps;
});
}
If I'm trying to do it synchronously by using a json object for example it will work:
var getApps = function(searchParam){
var appsInCategory = JSONOBJECT;
return appsInCategory
}
What can I do?
You can't return from a callback - see this canonical about the fundamental problem. Since you're working with Mongoose you can return a promise for it though:
var getApps = function(searchParam){
var appsInCategory = Model.find({ categories: searchParam});
return appsInCategory.exec().then(function (apps) {
return apps; // can drop the `then` here
});
}
Which would let you do:
getApps().then(function(result){
// handle result here
});

Return counter in db.collection.count()'s callback doesn't work, why?

I want to track the number of documents I have within a collection in a node.js server
using mongodb driver. I can insert, delete and update propperly but when I try to count, it works until I try to store that value, moment in which it returns nothing.
Here is my code:
var db_collection = db.collection('collection');
var countCollections = function () {
var response_count_collections = null;
db_mensajes.count(function(err,number_of_collections){
if (err){
console.log(err);
} else {
response_of_collections = number_of_collections;
console.log('Number of collections: '+number_of_collections);
}
});
return response_count_collections;
};
It logs the number_of_collections correctly but it doesn't return me the right value. In my example it returns null (how I defined my var response_count_collections;) and if I
try to return number_of_collections; within the db_collections.count()'s callback like this:
var db_collection = db.collection('collection');
var countCollections = function () {
var response_count_collections = null;
db_mensajes.count(function(err,number_of_collections){
if (err){
console.log(err);
} else {
console.log('Number of collections: '+number_of_collections);
return number_of_collections;
}
});
};
It returns me "undefined". Is there any way to achieve what I want?
Its because it returns the variable before the function is completely executed.
If you want it to be asynchronous then you will have to learn to control the flow of the program using some node modules like async.
Update:
Have a look at this question, it shows how to return value from an async function using callback correctly.

Making Mocha test work for my db module

I am new to Mocha, and only a little experience with Node/Express. My DbProvider module works perfectly (mongodb) when I am access it through my Express app. And now I want to test it. I have read the Mocha site and some tutorials I could find. But I have big trouble of finding an real-world example out there that I could follow (any links much appreciated!).
Here is my unsuccessful attempt to write a testfile:
var DbProvider = require('../db').DbProvider;
var assert = require('assert');
var dbProvider = new DbProvider('localhost', 27017, 'mydb');
var util = require('util');
console.log(util.inspect(dbProvider));
describe('DbProvider', function(){
describe('findAllNotes', function(){
it('should return some notes', function(){
dbProvider.findAllNotes({}, function (err, result){
assert(result.length > 0);
});
})
})
})
The output I get is this:
$ mocha
{}
✖ 1 of 1 test failed:
1) DbProvider findAllNotes should return some notes:
TypeError: Cannot call method 'collection' of undefined
at DbProvider.doOperation (/Users/frode/Node/json/db.js:46:11)
at DbProvider.findAllNotes (/Users/frode/Node/json/db.js:56:8)
at Context.<anonymous> (/Users/frode/Node/json/test/test.js:15:18)
(cutting out the rest)
It seems that I am unsuccessful to create the dbProvider. This works perfectly in my app... How can I make this work? (And perhaps also: Is the way I have set it up in general ok?)
Edit: Here is the db.js file:
// Database related
'use strict';
var MongoClient = require('mongodb').MongoClient;
var BSON = require('mongodb').BSONPure;
var ObjectID = require('mongodb').ObjectID;
var checkForHexRegExp = new RegExp("^[0-9a-fA-F]{24}$");
var Validator = require('validator').Validator
var fieldMaxLength = 1024;
//var util = require('util');
var DbProvider = function(host, port, database) {
var dbUrl = "mongodb://"+host+":"+port+"/"+database;
var self = this;
MongoClient.connect(dbUrl, function(err, db) {
self.db = db;
});
};
// Do some basic validation on the data we get from the client/user
var validateParams = function(params, callback) {
// Let´ do a quick general sanity check on the length on all fields
for(var key in params) {
if(params[key].length > fieldMaxLength) callback(new Error('Field ' + key + ' is too long.'));
}
// and the let us check some specific fields better
if (params._id) {
if(checkForHexRegExp.test(params._id)) {
// In case of '_id' we also need to convert it to BSON so that mongodb can use it.
params._id = new BSON.ObjectID(params._id);
} else {
var err = {error: 'Wrong ID format'};
}
}
if(err) callback(err);
}
// Generalized function to operations on the database
// Todo: Generalize even more when user authenication is implemented
DbProvider.prototype.doOperation = function(collection, operation, params, callback) {
validateParams(params, callback);
var operationCallback = function(err, result) {
callback(err, result);
};
this.db.collection(collection, function(err, collection) {
if(operation==='find') {
collection.find().toArray(operationCallback);
} else {
collection[operation](params, operationCallback);
}
});
}
DbProvider.prototype.findAllNotes = function(params, callback) {
this.doOperation('notes', 'find', params, callback);
};
DbProvider.prototype.findNoteById = function(params, callback) {
this.doOperation('notes', 'findOne', params, callback);
};
DbProvider.prototype.saveNote = function(params, callback) {
params.created_at = new Date();
this.doOperation('notes', 'save', params, callback);
};
DbProvider.prototype.deleteNote = function(params, callback) {
this.doOperation('notes', 'remove', params, callback);
};
DbProvider.prototype.findUser = function(params, callback) {
this.doOperation('users', 'findOne', params, callback);
};
exports.DbProvider = DbProvider;
SOLUTION:
After Benjamin told me to handle the async nature of mongodb connecting to the database, and inspired by his suggestion on how to adapt the code, I split the constructor function DbProvider into two parts. The first part, the constructor DbProvider now just saves the db-parameters into a variable. The second part, a new function, DbProvider.connect does the actual async connection. See below.
var DbProvider = function(host, port, database) {
this.dbUrl = "mongodb://"+host+":"+port+"/"+database;
};
DbProvider.prototype.connect = function(callback) {
var self = this;
MongoClient.connect(this.dbUrl, function(err, db) {
self.db = db;
callback();
});
};
So I can now make a Mocha test like this (and async tests also need the "Done" included, like you see in the code below):
var assert = require('assert');
var DbProvider = require('../db').DbProvider;
var dbProvider = new DbProvider('localhost', 27017, 'nki');
describe('DbProvider', function(){
describe('findAllNotes', function(){
it('should return some notes', function(done){
dbProvider.connect(function(){
dbProvider.findAllNotes({}, function (err, result){
assert(result.length > 0);
done();
});
});
})
})
})
Note that the acutal test ("should return some notes") is nothing to be proud of. What I wanted here was to get set up so I am able to test something. Now that I finally acutally can do that, I need to write good tests (something in the the line of having a test database, clear it, test insert a document, test search for a document, and so on...).
And in my Express app, I used to set up the database like this:
var DbProvider = require('./db').DbProvider;
// Setup db instance
var dbProvider = new DbProvider(
process.env.mongo_host || 'localhost',
process.env.mongo_port || 27017,
process.env.mongo_db || 'nki'
);
Now I do the same, but in addition, I call the new connect-function:
// Connect to db. I use (for now) 1 connection for the lifetime of this app.
// And I do not use a callback when connecting here (we do in the testing)
dbProvider.connect(function(){});
Benjamin actually pointed out that it may be ok but not the best practice to have the database set up like this in an Express app. But until I figure out what the best practice really is, I will leave this code as it is. Here is a couple of links reagarding the subject I found (but I have still not concluded of how I will solve it myself):
What's the best practice for MongoDB connections on Node.js? and
[node-mongodb-native] MongoDB Best practices for beginner
If you like, you are very welcome to follow/fork/whatever this project on github. My goal is to get it as production ready I can. The link is
https://github.com/frodefi/node-mongodb-json-server
MongoClient.connect is asynchronous.
From the docs:
callback (function) – this will be called after executing this method. The first parameter will contain the Error object if an error occured, or null otherwise. While the second parameter will contain the initialized db object or null if an error occured.
That means DbProvider.db isn't set yet in the test which is why you're getting undefined.
In here:
MongoClient.connect(dbUrl, function(err, db) {
self.db = db;
});
You're telling it "update self.db after the connection happened", which is at least one event loop tick after this one (but may be more). In your mocha code you're executing your .describe and .it methods right after creating your DbProvider instance which means it was not initialized yet.
I suggest that you re-factor DbProvider to return a callback instead of being a constructor function. Maybe something along the lines of:
var getDbProvider = function(host, port, database,callback) {
var dbUrl = "mongodb://"+host+":"+port+"/"+database;
MongoClient.connect(dbUrl, function(err, db) {
self.db = db;
callback(db);
});
};
Which also means moving all the DBProvider methods to an object (maybe the callback will return a dbprovider object and not just a db?).
Another bug solved by using Unit Tests :)
This is what I used: https://github.com/arunoda/mocha-mongo
It has set of testing helpers for mongodb

Resources