First, I'm sorry for the title, I couldn't mind up something better.
I thought I understand Node.js / KOA, at least the basics but now I'm starting to feel that I'm missing some fundamentals.
Take a look at the following code:
router.put("/",
parse,
async function (ctx, next) {
// do something
await next();
},
async function (ctx, next) {
// do something
await next();
},
async function (ctx, next) {
if (some condition) {
gm(imageBuffer)
.quality(80)
.write(profile_path, async function (err) {
gm(imageBuffer)
.resize(60, 60)
.quality(80)
.write(chat_path,async function (err) {
await next(); // HERE 1
});
});
} else {
await next();
}
// HERE 2
},
async function (ctx, next) {
responses.success(ctx, "Success");
}
);
So what this is all about. The ones that are familiar with KOA framework will immediately see what is going on here. Where my problem starts/ends is in the third async function. So what I'm trying to do here is some image manipulation (saving). gm is asynchronus, but as you can see from the code I'm using anonymous callback functions, and what I'm trying to achieve is that the last async function is being called when gm finishes through await next(); // HERE 1.
But what really happens is (from my understanding)... gm starts asynchronously.. and // HERE 2 is hit, and because there's nothing, end of function, KOA returns default 404 response. I simply can't understand why this is so and how to overcome this.
What I really want to happen is when callback finishes await next(); // HERE 1 gets called and I can return success response.
await next(); // HERE 1 of course gets called (eventually) but too late, because KOA already responds with 404.
If there is someone that is able and willing to explain what exactly is happening here, thank you.
As far as I see is that your aproach is not really following the async await pattern: The async function should the "await" the asynchronous part. This then needs to return a promise. So you have to encapulate your callback in a promise. Something like this could work (not testet, just to show the concept):
function gmAsync(){
return new Promise(function(resolve,reject){
gm(imageBuffer)
.quality(80)
.write(profile_path, async function (err) {
gm(imageBuffer)
.resize(60, 60)
.quality(80)
.write(chat_path,async function (err) {
resolve(....whatever....);
});
});
});
}
and then you async function could look like this:
async function (ctx, next) {
if (some condition) {
await gmAsync()
next()
} else {
await next();
}
},
...
Makes sense?
Related
I'm using mongoose with an async/await function to query the DB. A few calls to the api doesn't return the necessary data. Below is the controller code
exports.GetAllUrls = async function(req, res, next){
try {
var urlsArray = [];
await Url.find({uid: req.params.uid}, function (err, urls) {
urls.forEach(function(url){
urlsArray.push(url);
console.log(url);
});
});
console.log("Await called");
return res.status(200).json({reply: urlsArray});
} catch(err) {
console.log(err);
}
}
There are a few times where the "Await called" is logged before the url data is logged.
Node Request Log:
Await called
GET /api/screens/GetAllUrls/0WyaS0ePeaS54zz9cAgCUnxoE1i1 200 31.348 ms - 12
{ _id: 5b0ad7effa8e80800153fa04,
url: 'https://yahoo.com',
timestamp: 2018-05-27T16:31:10.638Z,
uid: '0WyaS0ePeaS54zz9cAgCUnxoE1i1',
__v: 0 }
As seen in the logs, the function seems to proceed before the await function is called, but my understanding is that execution is paused until the await is completed and returned. Anyone have any ideas why this is happening?
I don't know the specifics on that library or method, but I can tell you why it's not working.
"await" will pause only when the right hand side of the statement returns a "Promise" object. In your code example, it seems that the function takes a callback. Callbacks, though asynchronous, are not promises. Perhaps you can check that library's API docs to see if it can return a Promise instead of taking the callback?
You are mixing callbacks with async-await. Don't do that. Please study how they work: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function
Your route handler should be:
exports.GetAllUrls = async function(req, res, next){
try {
const urlsArray = await Url.find({uid: req.params.uid}).exec()
return res.status(200).json({reply: urlsArray});
} catch(err) {
console.log(err);
}
}
.find() returns a Query object: http://mongoosejs.com/docs/api.html#find_find
.exec() returns a Promise: http://mongoosejs.com/docs/api.html#query_Query-exec
This may be extremely stupid, but I haven't found much about this because I don't understand how I should search this.
I have a route handler that may call different functions depending on some request parameters, and I would like to know what's the best way to deal with errors inside the functions in order to pass errors to the error handling middleware.
Consider something like this:
router.get('/error/:error_id', (req, res, next) => {
my_function();
}
function my_function(){
// do something async, like readfile
var f = fs.readFile("blablabla", function (err, data) {
// would want to deal with the error
});
}
If an error occurs during fs.readFile, how do I pass the error to next to forward it to the error middleware? The only solution is to pass the next param to the function function my_function(next){...}?
In case the function didn't call any async I/O operation, a simple try/catch in the route handler would be ok (i suppose), like this:
router.get('/error/:error_id', (req, res, next) => {
try{
my_function();
} catch(e){
next(e);
};
}
function my_function(){
// do stuff
var f = fs.readFileSync("blablabla"); // possibly throws an error
}
Hope I make some sense.
You are totally correct that you should pass the next callback to my_function since fs.readFile is asynchronous.
router.get('/error/:error_id', (req, res, next) => {
my_function(next);
}
function my_function(next) {
fs.readFile("blablabla", function (err, data) {
if (err) {
next(err);
} else {
// Process the data
// Don't forget to call `next` to send respond the client
}
});
}
By the way, you cannot do
var f = fs.readFile(...)
because fs.readFile is asynchronous. The data should be handled within the callback.
I am trying to write a unit/integration test where I want to get a list of things in the database. For not it is only a GET, but these tests needs to extend to POST, PUT & DELETE.
The code I have thus far works fine, I can actually get data from the DB, but as soon as I try to stub out the function which is responsable for making the call to the DB, Mocha times out
1 failing
1) /account_types GET 200 List:
Error: timeout of 2000ms exceeded. Ensure the done() callback is being called in this test.
at null. (C:\Code\JS\general_admin_service\node_modules\mocha\lib\runnable.js:215:19)
I understand the done() callback isn't being called because the code is stuck somewhere, however, I do not understand what I am doing wrong.
I used the following references to get where I am:
Testing With Mocha, Sinon.js & Mocking Request
Lessons learned from unit testing with Sinon.JS
My code is as follows:
The Test:
'use strict';
var expect = require('chai').expect,
request = require('supertest'),
chance = require('chance').Chance(),
server = require('../server'),
sinon = require('sinon'),
select = require('../../helpers/data_access/select');
describe("/account_types", function () {
before(function(done){
sinon
.stub(select, "query_list")
.returns([{id: "test"}]);
done();
});
after(function(done){
select
.query_list
.restore();
done();
});
it('GET 200 List', function (done) {
request(server.baseURL)
.get('/api/v1/account_types')
.set('Accept', 'application/json')
.expect('Content-Type', 'application/json')
.expect(200)
.end(function (err, res) {
/* istanbul ignore if */
if (err)
return done(err);
expect(res.body).to.include.keys('result');
expect(res.body.result).to.not.be.null;
expect(res.body.result).to.not.be.undefined;
expect(res.body.result).to.be.an('Array');
expect(res.body.result.length).to.be.above(0);
//expect(select.query_list).to.have.been.calledOnce;
return done();
});
});
});
Restify endpoint:
var select = require('../helpers/data_access/select')
module.exports = function (server) {
var query = "..."
return select.query_list(res, next, db_config, query);
});
};
select.js:
var sql = require('mssql');
module.exports = {
query_list: function (res, next, config, sql_query) {
return query(res, next, config, sql_query, true);
},
query_single: function (res, next, config, sql_query) {
return query(res, next, config, sql_query, false);
}
};
function query(res, next, config, sql_query, isList) {
var connection = new sql.Connection(config);
connection.connect(function (err) {
if (err) {
return on_error(err, res);
}
var request = new sql.Request(connection);
request.query(sql_query, function (err, response) {
connection.close();
if (err) {
return on_error(err, res);
}
if (isList) {
return return_list(res, response, next);
} else {
return return_single(res, response, next);
}
});
});
}
function on_error(error, res, next) {
res.status(500).send(error);
return next();
}
function return_list(res, response, next) {
res.send({result: response});
return next();
}
function return_single(res, response, next) {
res.send({result: response[0]});
return next();
}
What I expect to happen is that because I stub out the query_list function, should I wish to put a console.log(res.body.result); after the expect's I have in place, I should see a return of [{id: "test"}], but it is obviously not getting to that point.
What am I doing wrong?
UPDATE: Added the full select.js file.
As you already make clear in the comments, it's difficult to test code that's deeply nested.
It's usually much better to work with callbacks or promises, so that each piece of your app will handle the part it's responsible for, but not (much) more. So your route handler will handle the request and the response. It's obviously okay to call other functions, like ones that perform database queries, but instead of letting those functions send back a response, you use callbacks that "call back" to the route handler with the query results.
Something like this:
server.get('/api/v1/account_types', function(req, res, next) {
select.query_list(QUERY, function(err, records) {
if (err) return next(err);
res.send({ results : records });
next();
});
});
In terms of using Sinon to test something like this: it really depends on the exact implementation. I can provide a quick example on how to stub the above usage of select.query_list, to make sure that the response contains the correct data.
The basic stub looks like this:
sinon.stub(select, 'query_list').yieldsAsync(null, [ { id : 'test' } ]);
What this does, is when select.query_list() gets call, it will call the first callback argument it receives (it does this by checking each argument to see which is a function) with the arguments null, [ { id : 'test' } ].
Those are the err and records arguments of the callback function passed in the handler. So you can use this to skip the database query entirely and pretend that the query yielded a particular array of records.
From there, res.send() gets called (which was the issue that you initially ran into: it didn't get called at all because it was being performed in a part of your app that wasn't getting called because of your stub) and you can check in your test if the resulting response data is as expected.
It becomes a bit more complicated if you want to stub a function deeper in the call stack, but with the correct Sinon tools (like .yields*, or using spies instead of stubs) it's usually not terribly difficult (provided that all the functions that you want to stub/spy are accessible, that is, exported).
I'm having a trouble testing a middleware in Express that utilises promise (using Q library).
Here's an example that resembles my case.
var httpMocks = require('node-mocks-http');
test('should return 404 status', function (done) {
var req = httpMocks.createRequest(),
var res = httpMocks.createResponse();
myMiddleware(req, res);
expect(req.statusCode).to.equal(404);
});
Let's say, myMiddleware makes a call to another module using promise that calls either next() on resolve or res.status(404).send() on reject.
I mocked the module for testing purpose.
how would I able to catch the end of myMiddleware on reject that does not return with next()?
Cheers!
You're going to have to change myMiddleware to either return a promise, or making a callback.
You could do something like this:
// myMiddleware now makes callback with potential error
function myMiddleware(req, res, callback) {
async.map(..., function(err, results){
callback(err);
});
}
Then in your test you can wait for the callback to test values:
test('should return 404 status', function (done) {
var req = httpMocks.createRequest(),
var res = httpMocks.createResponse();
myMiddleware(req, res, function(error) {
if(error) {
// Handle error for tests
}
// Test your values now
expect(req.statusCode).to.equal(404);
// All done
done();
});
});
In this example the callback from myMiddleware only returns an error, because apparently you're not concerned with anything else. In your case, we are mainly just using the callback as a trigger to test the values.
I'm using the Q library and async library in nodejs.
Here's an example of my code:
async.each(items, cb, function(item) {
saveItem.then(function(doc) {
cb();
});
}, function() {
});
saveItem is a promise. When I run this, I always get cb is undefined, I guess then() doesn't have access. Any ideas how to work around this?
Your issue doesn't lie with promises, it lies with your usage of async.
async.each(items, handler, finalCallback) applies handler to every item of the items array. The handler function is asynchronous, i.e. it is handed a callback, that it must call when it has finished its work. When all handlers are done, the final callback is called.
Here's how you'd fix your current issue:
var handler = function (item, cb) {
saveItem(item)
.then(
function () { // all is well!
cb();
},
function (err) { // something bad happened!
cb(err);
}
);
}
var finalCallback = function (err, results) {
// ...
}
async.each(items, handler, finalCallback);
However, you don't need to use async for this particular piece of code: promises alone fill this job quite nicely, especially with Q.all():
// Create an array of promises
var promises = items.map(saveItem);
// Wait for all promises to be resolved
Q.all(promises)
.then(
function () { // all is well!
cb();
},
function (err) { // something bad happened!
cb(err);
}
)