.then statements not executing sequentially - node.js

I have an application using Node.js/Express. Within this code I have the following promise designed to check if an email already exists in my (PostGres) database:
//queries.js
const checkEmail = function(mail) {
return new Promise(function(resolve, reject) {
pool.query('SELECT * FROM clients WHERE email = $1', [mail], function(error, results) {
if (error) {
reject(new Error('Client email NOT LOCATED in database!'));
} else {
resolve(results.rows[0]);
}
}) //pool.query
}); //new promise
}
In my 'main (server.js)' script file, I have a route which is called upon submission of a 'signup' form. When the post to this route is processed...I run the script above to check if the passed email address is already located in the database, along with various other 'hashing' routines:
My code is as follows:
//server.js
const db = require('./queries');
const traffic = require('./traffic');
const shortid = require('shortid');
...
app.post('/_join_up', function(req, res) {
if (!req.body) {
console.log('ERROR: req.body has NOT been returned...');
return res.sendStatus(400)
}
var newHash, newName;
var client = req.body.client_email;
var creds = req.body.client_pword;
var newToken = shortid.generate();
var firstname = req.body.client_alias;
db.sanitation(client, creds, firstname).then(
function(direction) {
console.log('USER-SUPPLIED DATA HAS PASSED INSPECTION');
}
).then(
db.checkEmail(client).then(
function(foundUser) {
console.log('HEY THERE IS ALREADY A USER WITH THAT EMAIL!', foundUser);
},
function(error) {
console.log('USER EMAIL NOT CURRENTLY IN DATABASE...THEREFORE IT IS OK...');
}
)).then(
traffic.hashPassword(creds).then(
function(hashedPassword) {
console.log('PASSWORD HASHED');
newHash = hashedPassword;
},
function(error) {
console.log('UNABLE TO HASH PASSWORD...' + error);
}
)).then(
traffic.hashUsername(firstname).then(
function(hashedName) {
console.log('NAME HASHED');
newName = hashedName;
},
function(error) {
console.log('UNABLE TO HASH NAME...' + error);
}
)).then(
db.createUser(client, newName, newHash, newToken).then(
function(data) {
console.log('REGISTERED A NEW CLIENT JOIN...!!!');
res.redirect('/landing'); //route to 'landing' page...
},
function(error) {
console.log('UNABLE TO CREATE NEW USER...' + error);
}
))
.catch(function(error) {
console.log('THERE WAS AN ERROR IN THE SEQUENTIAL PROCESSING OF THE USER-SUPPLIED INFORMATION...' + error);
res.redirect('/');
});
}); //POST '_join_up' is used to register NEW clients...
My issue is the '.then' statements do not appear to run sequentially. I was under the impression such commands only run one after the other...with each running only when the previous has completed. This is based upon the logs which show the readout of the 'console.log' statements:
USER-SUPPLIED DATA HAS PASSED INSPECTION
PASSWORD HASHED
NAME HASHED
UNABLE TO CREATE NEW USER...Error: Unable to create new CLIENT JOIN!
USER EMAIL NOT CURRENTLY IN DATABASE...THEREFORE IT IS OK...
As mentioned previously, I am under the impression the '.then' statements should run synchronously, therefore the last statement ("USER EMAIL NOT CURRENTLY IN DATABASE...THEREFORE IT IS OK...") should in fact be after the first...before the "PASSWORD HASHED" according to the layout of the '.then' statements. Is this normal behavior...or do I have an error in my code?
Sorry for my confusion however I find '.then' statements and promises to be somewhat confusing for some reason. I thank you in advance.

TLDR - You must pass a function reference to .then() so the promise infrastructure can call that function later. You are not doing that in several places in your code.
A more specific example from your code:
You have several structures like this:
.then(db.createUser().then())
This is incorrect. This tells the interpreter to run db.createUser() immediately and pass its return result (a promise) to .then(). .then() will completely IGNORE anything you pass is that is not a function reference and your promises will not be properly chained.
Instead, you must pass a function reference to .then() something like this (not sure what execution logic you actually want):
.then(() => { return db.createUser.then()})
Then main point here is that if you're going to sequence asynchronous operations, then you must chain their promises which means you must not execute the 2nd until the first calls the function you pass to .then(). You weren't passing a function to .then(), you were executing a function immediately and then passing a promise to .then(p) which was completely ignored by .then() and your function was executed before the parent promise resolved.
FYI, sequencing a bunch of asynchronous operations (which it appears you are trying to do here) can take advantage of await instead of .then() and end up with much simpler looking code.

Related

Asynchronous CRUD operations with express

I have a basic CRUD application using html forms, nodejs/express and mongodb. I have been learning about synchronous vs asynchronous code via callbacks, promises, and async/await and to my understanding for a crud application you would want the operations to be asynchronous so multiple users can do the operations at the same time. I am trying to implement aync/await with my express crud operations and am not sure if they are executing synchronously or asynchronously.
Here is my update function, which allows a user to type in the _id of the blog they want to change, then type in a new title and new body for the blog and submit it. In its current state, to my knowledge it is executing synchronously:
app.post('/update', (req, res) => {
const oldValue = { _id: new mongodb.ObjectId(String(req.body.previousValue)) }
const newValues = { $set: { blogTitle: req.body.newValue, blogBody: req.body.newValue2 } }
db.collection("miscData").updateOne(oldValue, newValues, function (err, result) {
if (err) throw err;
console.log("1 document updated");
res.redirect('/')
});
})
The way in which I was going to change this to asynchronous was this way:
app.post('/update', async (req, res) => {
const oldValue = { _id: new mongodb.ObjectId(String(req.body.previousValue)) }
const newValues = { $set: { blogTitle: req.body.newValue, blogBody: req.body.newValue2 } }
await db.collection("miscData").updateOne(oldValue, newValues, function (err, result) {
if (err) throw err;
console.log("1 document updated");
res.redirect('/')
});
})
Both blocks of code work, however I am not sure if the second block of code is doing what I am intending it to do, which is allow a user to update a blog without blocking the call stack, or if the second block of code would only make sense if I was running more functions after the await. Does this achieve the intended purpose, if not how could/should I do that?
db.collection(...).updateOne is always asynchronous, so you need not worry that a long-running database operation might block your application. There are two ways how you can obtain the asynchronous result:
With a callback function
db.collection(...).updateOne(oldValues, newValues, function(err, result) {...});
console.log("This happens synchronously");
The callback function with the two parameters (err, result) will be called asynchronously, after the database operation has completed (and after the console.log). Either err contains a database error message or result contains the database result.
With promises
try {
var result = await db.collection(...).updateOne(oldValues, newValues);
// Do something with result
} catch(err) {
// Do something with err
}
console.log("This happens asynchronously");
The updateOne function without a callback function as third parameter returns a promise that must be awaited. The statements that do something with result will be executed asynchronously, after the database operation has successfully completed. If a database error occurs, the statements in the catch block are executed instead. In either case (success or error), the console.log is only executed afterwards.
(If updateOne does not have a two-parameter version, you can write
var result = await util.promisify(db.collection(...).updateOne)(oldValues, newValues);
using util.promisify.)
Your second code snippet contains a mixture of both ways (third parameter plus await), which does not make sense.

How to store mongodb output in variable using nodejs in lambda function?

In a lambda function, I have the following code:
var user;
exports.handler = function uploadToS3(event, context, callback) {
var name = event["username"];
MongoClient.connect(uri, { useNewUrlParser: true }, (error, client) => {
if (error) return 1; // Checking the connection
db = client.db(databasename);
db.collection("user_profile").findOne({ username: name }, function(
err,
result
) {
if (err) throw err;
user = result._id;
console.log(user); // 1st console.log
});
});
console.log(user); //2nd console.log
};
In the above code, I have declared user as a global variable. In 1st console.log it will display the value but in 2nd console.log it will undefined. find the below output of lambda function.
Function Logs:
2019-08-23T15:23:34.610Z 83141f62-f840-4e52-9440-35f3be7b0dc8
5d5eaa9f921ed00001ee1c3f
2019-08-23T15:23:34.192Z 83141f62-f840-4e52-9440-35f3be7b0dc8
undefined
How can I get a value in the second case?
The problem is not so much storing the mongodb output into a variable, as it is a synchronous vs asynchronous behavior. Javascript by design is synchronous, but has capability to handle asynchronous tasks. The method that performs the mongo query is asynchronous. Read: Javascript calls findOne(), this returns an 'pending' promise, then your script continues to call console.log(user) - which is still undefined. When the request from MongoDB comes back, javascript resolves the promise and executes any further actions and/or callbacks.
The second console.log comes back and is evaluated BEFORE the mongo client returns a response and assigns a new value to your variable. If you look at the timestamp of the responses, the undefined one comes back before the one with the value. It looks like you are using mongoose, which should return a promise and you can try putting that second call inside a .then, or a .done block. e.g:
var user;
exports.handler = function uploadToS3(event, context, callback) {
var name = event["username"];
MongoClient.connect(uri, { useNewUrlParser: true }, (error, client) => {
if (error) return 1; // Checking the connection
db = client.db(databasename);
db.collection("user_profile").findOne({ username: name }, function(
err,
result
) {
if (err) throw err;
user = result._id;
console.log(user); // 1st console.log
})
.done(function(){
console.log(user); //2nd console.log
});
});
};
If not using mongoose... make your own promise, or use a callback, or just try Mongoose (it rocks!) :)
*note that I put the .done after the findOne(), but I believe you could attach a .done() to the .connect() as well. (Don't quote me on that. You would have to test it, see when that promise resolves exactly)
Additionally, I would suggest storing this value outside of your lambda somehow. You might not get the same container bootstrapped for each lambda execution. You could have some issues with this down the line.
Check out:
AWS Lambda caching issues with Global Variables - https://medium.com/tensult/aws-lambda-function-issues-with-global-variables-eb5785d4b876
Improving Performance From Your Lambda Function From the Use of Global Variables - https://blog.ruanbekker.com/blog/2018/08/27/improving-performance-from-your-lambda-function-from-the-use-of-global-variables/
AWS Lambda best practices - https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html

Recover an object requested with MongoDB Driver on node JS

I try to recover object from a mongo DB database, in a node JS file and it doesn't work.
In a file called db.js , i have made the following code :
var MongoClient = require('mongodb').MongoClient;
module.exports = {
FindinColADSL: function() {
return MongoClient.connect("mongodb://localhost/sdb").then(function(db) {
var collection = db.collection('scollection');
return collection.find({"type" : "ADSL"}).toArray();
}).then(function(items) {
return items;
});
}
};
And, I try to use it in the file server.js :
var db = require(__dirname+'/model/db.js');
var collection = db.FindinColADSL().then(function(items) {
return items;
}, function(err) {
console.error('The promise was rejected', err, err.stack);
});
console.log(collection);
In the result I have "Promise { }". Why ?
I just want to obtain an object from the database in order to manipulate it in the others functions situated in the server.js file.
Then then function called on promises returns a promise. If a value is returned within a promise, the object the promise evaluates to is another promise which resolves to the value returned. Take a look at this question for a full explanation of how it works.
If you want to verify that your code is successfully getting the items, you will have to restructure your code to account for the structure of promises.
var db = require(__dirname+'/model/db.js');
var collection = db.FindinColADSL().then(function(items) {
console.log(items);
return items;
}, function(err) {
console.error('The promise was rejected', err, err.stack);
});
That should log your items after they are retrieved from the database.
Promises work this way to make working asynchronously more simple. If you put more code below your collection code, it would run at the same time as your database code. If you have other functions within your server.js file, you should be able to call them from within the body of your promises.
As a rule, remember a promise will always return a promise.
The callback functions created in the then() are asynchronous, thus making the console.log command execute before the promise is even resolved. Try placing it inside the callback function instead like below:
var collection = db.FindinColADSL().then(function(items) {
console.log(items)
return items;
}, function(err) {
console.error('The promise was rejected', err, err.stack);
});
Or, for the sake of another example using the logger functions themselves as the callbacks, and showing that the last console.log call will actually be called before the others.
db.findinColADSL()
.then(console.log)
.catch(console.error)
console.log('This function is triggered FIRST')

Why am I getting a warning when using express app.param to pre-load object with sequelize?

I'm using express app.param to load objects from db (app.param('userId', users.userById);):
exports.userById = function (req, res, next, id) {
return user.findOne({
where: { id: id }
}).then(function (result) {
req.user = result;
next();
}).catch(function (error) {
next(error);
});
};
After that I update the loaded object with the following code.
exports.update = function (req, res) {
var user = req.user;
return user.update({
//update properties
}).then(function () {
res.end();
}).catch(function (error) {
//error handling
});
};
For some reason I get the warning that "a promise was created in a handler but was not returned from it".
I can't see why, but that always happen when I use a routing parameter that uses sequelize before making the actual changes to the database.
What is the correct way to do this?
I'm using sequelize v3.23.3 with Postgres.
EDIT
I changed the code to a more simple example that throws the same warning.
If you forget to return that request promise, the next handler executes immediately with an argument of undefined - which is completely valid for the Promises/A+ spec, but I don't think that it is what you are looking for.
See How to execute code after loop completes for solutions.

Testing Express and Mongoose with Mocha

I'm trying to test my REST API endpoint handlers using Mocha and Chai, the application was built using Express and Mongoose. My handlers are mostly of the form:
var handler = function (req, res, next) {
// Process the request, prepare the variables
// Call a Mongoose function
Model.operation({'search': 'items'}, function(err, results) {
// Process the results, send call next(err) if necessary
// Return the object or objects
return res.send(results)
}
}
For example:
auth.getUser = function (req, res, next) {
// Find the requested user
User.findById(req.params.id, function (err, user) {
// If there is an error, cascade down
if (err) {
return next(err);
}
// If the user was not found, return 404
else if (!user) {
return res.status(404).send('The user could not be found');
}
// If the user was found
else {
// Remove the password
user = user.toObject();
delete user.password;
// If the user is not the authenticated user, remove the email
if (!(req.isAuthenticated() && (req.user.username === user.username))) {
delete user.email;
}
// Return the user
return res.send(user);
}
});
};
The problem with this is that the function returns as it calls the Mongoose method and test cases like this:
it('Should create a user', function () {
auth.createUser(request, response);
var data = JSON.parse(response._getData());
data.username.should.equal('some_user');
});
never pass as the function is returning before doing anything. Mongoose is mocked using Mockgoose and the request and response objects are mocked with Express-Mocks-HTTP.
While using superagent and other request libraries is fairly common, I would prefer to test the functions in isolation, instead of testing the whole framework.
Is there a way to make the test wait before evaluating the should statements without changing the code I'm testing to return promises?
You should use an asynchronous version of the test, by providing a function with a done argument to it.
For more details refer to http://mochajs.org/#asynchronous-code.
Since you don't want to modify your code, one way to do that could be by using setTimeout in the test to wait before to call done.
I would try something like this:
it('Should create a user', function (done) {
auth.createUser(request, response);
setTimeout(function(){
var data = JSON.parse(response._getData());
data.username.should.equal('some_user');
done();
}, 1000); // waiting one second to perform the test
});
(There might be better way)
Apparently, express-mocks-http was abandoned a while ago and the new code is under node-mocks-http. Using this new library it is possible to do what I was asking for using events. It's not documented but looking at the code you can figure it out.
When creating the response object you have to pass the EventEmitter object:
var EventEmitter = require('events').EventEmitter;
var response = NodeMocks.createResponse({eventEmitter: EventEmitter});
Then, on the test, you add a listener to the event 'end' or 'send' as both of them are triggered when the call to res.send. 'end' covers more than 'send', in case you have calls other than res.send (for example, res.status(404).end().
The test would look something like this:
it('Should return the user after creation', function (done) {
auth.createUser(request, response);
response.on('send', function () {
var data = response._getData();
data.username.should.equal('someone');
data.email.should.equal('asdf2#asdf.com');
done();
});
});

Resources