Recently I switched from using callbacks to using promise in my rest api express app.
But I'm having trouble with unit testing routes/controller with async behaviour of the promise. Here is the sample code that needs to be unit tested.
var handler = function (req, res, next) {
var query = {},
var options = {
sort: { updatedAt: -1 },
limit: 10
};
if (req.query.before) {
query.updatedAt = { $lt: req.query.before };
}
// User.findAsync returns bluebird promise
User.findAsync(query, null, options).then(function (user) {
res.json(user);
}).catch(function (e) {
next(e);
});
}
router.get('/api/users', handler);
My approach to test above code was to spy on req, next, and User.findAsync and check if they are called with correct arguments. But because of async behaviour of the promise, I was having trouble to check if res.json or next are get called.
I've tried to stub findAsync to return resolved promise (Promise.resolve(user)). but still then callback is executed asynchronously.
I'm not sure if I'm on the right track for testing express application.
What is good strategy to test this kind of code in good separation?
I've also heard about using supertest.
But for me, Using supertest to test from http end point feels like more of integration testing which is not unit testing and is quite expensive.
Also, In general, I would like to know if it is good practice to try to cover all of the code with unit testing (models, controller, middleware, etc) and what's good strategies or techniques of doing that. Or If it is just good enough to test http end points with super test.
If your method being tested doesn't return a promise then you can't use the promise syntax in Mocha. You can test your method the same way you'd test any other asynchronous method - with done as a parameter of it. Let's say we want to test your handler function:
var handler = function (req, res, next) {
//...
User.findAsync(query, null, options).then(function (user) {
res.json(user);
}).catch(function (e) {
next(e);
});
}
We can write a test as such:
describe("The handler", function(){
it("calls res.json", function(done){ // note the done argument
handler({query: {before: 5}, // mock request
{json: done} // res.json calls our `done` param of this function
function(){ throw new Error("error called"); });
});
});
Note that we mocked the request, response and the next handler. Our mocked response has a json method that lets the test know it is complete (this can be a function if you want to make assertions inside it) and if next is called instead we throw to signal it's not something that was supposed to happen.
Related
These two middleware functions behave differently and I cannot figure out why:
Here, the error will get trapped by try/catch:
router.get('/force_async_error/0', async function (req, res, next) {
try{
await Promise.reject(new Error('my zoom 0'));
}
catch(err){
next(err);
}
});
But here, the error will not get trapped by try/catch:
router.get('/force_async_error/1', async function (req, res, next) {
await Promise.reject(new Error('my zoom 1'));
});
I thought Express wrapped all middleware functions with try/catch, so I don't see how it would behave differently?
I looked into the Express source, and the handler looks like:
Layer.prototype.handle_request = function handle(req, res, next) {
var fn = this.handle;
if (fn.length > 3) {
// not a standard request handler
return next();
}
try {
fn(req, res, next); // shouldn't this trap the async/await error?
} catch (err) {
next(err);
}
};
so why doesn't the try/catch there capture the thrown error?
I'm going to add an answer here even though you've already accepted another one because I think what's going on here can be explained better and this will help others attempting to understand this.
In your code here:
router.get('/force_async_error/1', async function (req, res, next) {
await Promise.reject(new Error('my zoom 1'));
});
Let's discuss what is going on:
First, you declared the callback as async which you had to do in order to use await in it. An async function tells the interpreter to do several important things.
1. An async function always returns a promise. The resolved value of the promise will be whatever the function returns.
2. An async function is internally wrapped with a try/catch. If any exceptions are thrown in the top level scope of the function code, then those exceptions will be caught and will automatically reject the promise that the function returns.
3. An async function allows you to use await. This is an indicator to the interpreter that it should implement and allow the await syntax inside the function. This is tied to the previous two points above which is why you can't use await in just any 'ol function. Any uncaught rejections from await will also reject the promise that the function returns.
It's important to understand that while the async/await syntax allows you to kind of program with exceptions and try/catch like synchronous code, it isn't exactly the same thing. The function is still returning a promise immediately and uncaught exceptions in the function cause that promise to get rejected at some time later. They don't cause a synchronous exception to bubble up to the caller. So, the Express try/catch won't see a synchronous exception.
But here, the error will not get trapped by try/catch
I thought Express wrapped all middleware functions with try/catch, so I don't see how it would behave differently?
so why doesn't the try/catch [in Express] there capture the thrown error?
This is for two reasons:
The rejected promise is not a synchronous throw so there's no way for Express to catch it with a try/catch. The function just returns a rejected promise.
Express is not looking at the return value of the route handler callback at all (you can see that in the Express code you show). So, the fact that your async function returns a promise which is later rejected is just completely ignored by Express. It just does this fn(req, res, next); and does not pay attention to the returned promise. Thus the rejection of the promise falls on deaf ears.
There is a somewhat Express-like framework called Koa that uses promises a lot and does pay attention to returned promises and which would see your rejected promise. But, that's not what Express does.
If you wanted some Koa-type functionality in Express, you could implement it yourself. In order to leave other functionality undisturbed so it can work normally, I'll implement a new method called getAsync that does use promises:
router.getAsync = function(...args) {
let fn = args.pop();
// replace route with our own route wrapper
args.push(function(req, res, next) {
let p = fn(req, res, next);
// if it looks like a promise was returned here
if (p && typeof p.catch === "function") {
p.catch(err => {
next(err);
});
}
});
return router.get(...args);
}
You could then do this:
router.getAsync('/force_async_error/1', async function (req, res, next) {
await Promise.reject(new Error('my zoom 1'));
});
And, it would properly call next(err) with your error.
Or, your code could even just be this:
router.getAsync('/force_async_error/1', function (req, res, next) {
return Promise.reject(new Error('my zoom 1'));
});
P.S. In a full implementation, you'd probably make async versions of a bunch of the verbs and you'd implement it for middleware and you'd put it on the router prototype. But, this example is to show you how that could work, not to do a full implementation here.
This is because the call is asynchronous, take this code :
try {
console.log('Before setTimeout')
setTimeout(() => {
throw new Error('Oups')
})
console.log('After setTimeout')
}
catch(err) {
console.log('Caught', err)
}
console.log("Point of non-return, I can't handle anything anymore")
If you run it you should see that the error is triggered after Point of non-return.
When we're at the throw line it's too late, we're outside of try/catch. At this moment if an error is thrown it'll be uncaught.
You can work around this by using async/await in the caller (doesn't matter for the callee), ie :
void async function () {
try {
console.log('Before setTimeout')
await new Promise((resolve, reject) =>
setTimeout(() => {
reject(new Error('Oups'))
})
)
console.log('After setTimeout')
}
catch(err) {
console.log('Caught', err.stack)
}
console.log("Point of non-return, I can't handle anything anymore")
}()
Finally, this means that for Express to handle async errors you would need to change the code to :
async function handle(req, res, next) {
// [...]
try {
await fn(req, res, next); // shouldn't this trap the async/await error?
} catch (err) {
next(err);
}
}
A better workaround:
Define a wrap function like this :
const wrap = fn => (...args) => Promise
.resolve(fn(...args))
.catch(args[2])
And use it like this :
app.get('/', wrap(async () => {
await Promise.reject('It crashes!')
}))
Neither of these really answer the question, which if I understand correctly is:
Since the async/await syntax lets you handle rejected "awaits" with non-async style try/catch syntax, why doesn't a failed "await" get handled by Express' try/catch at the top level and turned into a 500 for you?
I believe the answer is that whatever function in the Express internals that calls you would also have to be declared with "async" and invoke your handler with "await" to enable async-catching try/catch to work at that level.
Wonder if there's a feature request for the Express team? All they'd need to add is two keywords in two places. If success, do nothing, if exception hand off to the error handling stack.
Beware that if you don't await or return the promise, it has nothing to do with express - it just crashes the whole process.
For a general solution for detached promise rejections:
https://stackoverflow.com/a/28709667
Copied from above answer:
process.on("unhandledRejection", function(reason, p){
console.log("Unhandled", reason, p); // log all your errors, "unsuppressing" them.
//throw reason; // optional, in case you want to treat these as errors
});
I'm looking for a solution to waiting for an event to happen before sending a HTTP response.
Use Case
The idea is I call a function in one of my routes: zwave.connect("/dev/ttyACM5"); This function return immediately.
But there exists 2 events that notice about if it succeed or fail to connect the device:
zwave.on('driver ready', function(){...});
zwave.on('driver failed', function(){...});
In my route, I would like to know if the device succeed or fail to connect before sending the HTTP response.
My "solution"
When an event happen, I save the event in a database:
zwave.on('driver ready', function(){
//In the database, save the fact the event happened, here it's event "CONNECTED"
});
In my route, execute the connect function and wait for the event to
appear in the database:
router.get('/', function(request, response, next) {
zwave.connect("/dev/ttyACM5");
waitForEvent("CONNECTED", 5, null, function(){
response.redirect(/connected);
});
});
// The function use to wait for the event
waitForEvent: function(eventType, nbCallMax, nbCall, callback){
if(nbCall == null) nbCall = 1;
if(nbCallMax == null) nbCallMax = 1;
// Looking for event to happen (return true if event happened, false otherwise
event = findEventInDataBase(eventType);
if(event){
waitForEvent(eventType, nbCallMax, nbCall, callback);
}else{
setTimeout(waitForEvent(eventType, callback, nbCallMax, (nbCall+1)), 1500);
}
}
I don't think it is a good practice because it iterates calls over the database.
So what are your opinions/suggestions about it?
I've gone ahead and added the asynchronous and control-flow tags to your question because at the core of it, that is what you're asking about. (As an aside, if you're not using ES6 you should be able to translate the code below back to ES5.)
TL;DR
There are a lot of ways to handle async control flow in JavaScript (see also: What is the best control flow module for node.js?). You are looking for a structured way to handle it—likely Promises or the Reactive Extensions for JavaScript (a.k.a RxJS).
Example using a Promise
From MDN:
The Promise object is used for asynchronous computations. A Promise represents a value which may be available now, or in the future, or never.
The async computation in your case is the computation of a boolean value describing the success or failure to connect to the device. To do so, you can wrap the call to connect in a Promise object like so:
const p = new Promise((resolve) => {
// This assumes that the events are mutually exclusive
zwave.connect('/dev/ttyACM5');
zwave.on('driver ready', () => resolve(true));
zwave.on('driver failed', () => resolve(false));
});
Once you have a Promise representing the state of the connection, you can attach functions to its "future" value:
// Inside your route file
const p = /* ... */;
router.get('/', function(request, response, next) {
p.then(successful => {
if (successful) {
response.redirect('/connected');
}
else {
response.redirect('/failure');
}
});
});
You can learn more about Promises on MDN, or by reading one of many other resources on the topic (e.g. You're Missing the Point of Promises).
Have you tried this? From the look of it, your zwave probably have already implemented an EventEmmiter, you just need to attach a listener to it
router.get('/', function(request, response, next) {
zwave.connect("/dev/ttyACM5");
zwave.once('driver ready', function(){
response.redirect(/connected);
});
});
There is a npm sync module also. which is used for synchronize the process of executing the query.
When you want to run parallel queries in synchronous way then node restrict to do that because it never wait for response. and sync module is much perfect for that kind of solution.
Sample code
/*require sync module*/
var Sync = require('sync');
app.get('/',function(req,res,next){
story.find().exec(function(err,data){
var sync_function_data = find_user.sync(null, {name: "sanjeev"});
res.send({story:data,user:sync_function_data});
});
});
/*****sync function defined here *******/
function find_user(req_json, callback) {
process.nextTick(function () {
users.find(req_json,function (err,data)
{
if (!err) {
callback(null, data);
} else {
callback(null, err);
}
});
});
}
reference link: https://www.npmjs.com/package/sync
I have some mocha tests I run with Nodejs to test a web server.
Many of the tests should cause the server to return an error, e.g. 400 Bad Request.
Currently the tests are peppered with many copies of the following code:
it('should respond with 400 (Bad Request)', function (){
expect(httpResponse.statusCode).to.equal(httpstatus.BAD_REQUEST);
});
Here's a simplified pseudocode example:
describe('When passing bad JSON data', function(){
var response
before(function(done){
callUrlToInsert(url, badJson, function(err, resp){
response = resp
done()
}
}
it('should respond with 400 (Bad Request)', function (){
expect(httpResponse.statusCode).to.equal(httpstatus.BAD_REQUEST)
})
}
This bugs me because as a programmer I avoid duplicate code wherever possible.
However, putting this into a function does not work:
function verifyItReturnedBadRequest400(httpResponse)
{
it('should respond with 400 (Bad Request)', function (){
expect(httpResponse.statusCode).to.equal(httpstatus.BAD_REQUEST);
});
}
because the call to it() doesn't test the assertion right then; my [limited] understanding is that it() adds the closure to the list of tests. So by the time that check is done, the httpResponse variable has gone out of scope. (I don't understand why that is the case, because in both cases there is a call to it(); why would it matter that in one case it's inside another level of function call? I'm probably missing something with regard to Javascript scoping.)
Is there a common way to avoid all this duplicate code? Or is everyone out there duplicating all their assertion code everywhere? This is my first foray into Mocha so I am probably missing something obvious.
Also, bonus points for explaining why doesn't the function approach work?
Thanks!
There is an article on wiki about this.
https://github.com/mochajs/mocha/wiki/Shared-Behaviours
I guess you have some bugs in your test. Placing it() into wrapper function works fine. Here's a small working demo.
'use strict';
const assert = require('assert');
const xEqualsOne = () => {
it('should be equal 1', () => {
assert.equal(this.x, 1);
});
};
describe('async number', () => {
this.x = 0;
before(done => {
this.x++
setTimeout(done, 100);
});
xEqualsOne();
});
I guess your code looks something like this:
describe('When passing bad JSON data', function(){
var response
before(function(done){
callUrlToInsert(url, badJson, function(err, resp){
response = resp
done()
}
}
verifyItReturnedBadRequest400(httpResponse)
}
Think about it like this:
it() creates a test.
All the calls to it happen before any tests are actually run (you have to create tests before you run them)
The function passed to `before' is run after the tests have been created, but before they are run.
verifyItReturnedBadRequest400 calls it, to create a test, but you're passing in httpResponse right then before any tests have run, so before hasn't run yet either.
You could continue to use that sort of pattern, but you'll need to put the httpresponse in a container so you can pass a reference to it:
describe('When passing bad JSON data', function(){
var data = {};
before(function(done){
callUrlToInsert(url, badJson, function(err, resp){
data.response = resp
done()
}
}
verifyItReturnedBadRequest400(data)
}
and then your verifyItReturnedBadRequest400 becomes:
function verifyItReturnedBadRequest400(data) {
it('should respond with 400 (Bad Request)', function (){
expect(data.response.statusCode).to.equal(httpstatus.BAD_REQUEST);
});
}
I've got a MEAN app and I'm trying to get tests to work on the node side. Async events are wrapped in promises, which are consumed in the controller. I failed at testing the controller :(
The controller I'm trying to test:
ProjectController.prototype.getAll = function(req, res, next) {
req.dic.subjectRepository
.getById(req.params.subjectId)
.then(function(subject) {
res.json(subject.projects);
}, function(err) {
return res.status(404).send('Subject does not exist.' + err);
});
};
The subjectRepository is our data source, which returns a promise (mpromise because under the hood we're using mongoose, but it shouldn't matter):
So in our test we tried mocking the request (we're injecting our dependency injection container from a middleware into the req) and response (the test succeeds if response.json() has been called with the subjects we tried to fetch) and our subjectRepository. We used bluebird (although I tried others out of frustration) to create fake promises for our mocked subjectRepository:
describe('SubjectController', function() {
'use strict';
var Promise = require('bluebird');
it('gets all existing subjects', function() {
// -------------------------------------
// subjectRepository Mock
var subjectRepository = {
getAll: function() {},
};
var subjectPromise = Promise.resolve([
{name: 'test'},
{name: 'test2'},
]);
spyOn(subjectRepository, 'getAll').andReturn(subjectPromise);
// -------------------------------------
// request mock
var req = {
dic: {
subjectRepository: subjectRepository,
},
};
// -------------------------------------
// response mock
var res = {
json: function() {},
send: function() {},
};
spyOn(res, 'json');
// -------------------------------------
// actual test
var subjectController = new (require('../../../private/controllers/SubjectController'))();
subjectController.getAll(req, res);
// this succeeds
expect(subjectRepository.getAll).toHaveBeenCalled();
// this fails
// expect(res.json).toHaveBeenCalled();
});
});
Question: How do I make the test run the expect() AFTER the promise succeeded?
Node v0.12
The code is on GitHub for anyone who's interested: https://github.com/mihaeu/fair-projects
And maybe I should mention that the controller is called from the router:
// router handles only routing
// and controller handles data between view and model (=MVC)
subjectRouter.get('/:subjectId', subjectController.get);
I got this to work by changing our controllers to hand down the promises, but I'm not sure this is what we want. Isn't there a way to get my approach to work?
it('gets all existing subjects', function(done) {
// ...
var subjectController = new (require('../../../private/controllers/SubjectController'))();
subjectController.getAll(req, res).then(function() {
expect(res.json).toHaveBeenCalledWith(testSubjects); // success
}).finally(done);
expect(subjectRepository.getAll).toHaveBeenCalled(); // success
}
Your code makes the mistake of mixing business logic with front facing routing.
If your getAll did not touch the request and response object, it would look something like this:
ProjectController.prototype.getAll = function(subjectId) {
return req.dic.subjectRepository.getById(subjectId).then(function(subject){
return subject.projects;
});
};
Now, it is no longer related to the request response life cycle or in charge of logic, testing it is trivial by:
it("does foo", function(){
// resolve to pass the test, reject otherwise, mocha or jasmine-as-promised
return controller.getAll(152).then(...)
});
That would make your actual handler look like:
app.get("/projects", function(req, res){
controller.getAll(req.params.subjectId).then(function(result){
res.json(result);
}, function(){
res.status(404).send("...");
});
});
In my node application I'm using mocha to test my code. While calling many asynchronous functions using mocha, I'm getting timeout error (Error: timeout of 2000ms exceeded.). How can I resolve this?
var module = require('../lib/myModule');
var should = require('chai').should();
describe('Testing Module', function() {
it('Save Data', function(done) {
this.timeout(15000);
var data = {
a: 'aa',
b: 'bb'
};
module.save(data, function(err, res) {
should.not.exist(err);
done();
});
});
it('Get Data By Id', function(done) {
var id = "28ca9";
module.get(id, function(err, res) {
console.log(res);
should.not.exist(err);
done();
});
});
});
You can either set the timeout when running your test:
mocha --timeout 15000
Or you can set the timeout for each suite or each test programmatically:
describe('...', function(){
this.timeout(15000);
it('...', function(done){
this.timeout(15000);
setTimeout(done, 15000);
});
});
For more info see the docs.
I find that the "solution" of just increasing the timeouts obscures what's really going on here, which is either
Your code and/or network calls are way too slow (should be sub 100 ms for a good user experience)
The assertions (tests) are failing and something is swallowing the errors before Mocha is able to act on them.
You usually encounter #2 when Mocha doesn't receive assertion errors from a callback. This is caused by some other code swallowing the exception further up the stack. The right way of dealing with this is to fix the code and not swallow the error.
When external code swallows your errors
In case it's a library function that you are unable to modify, you need to catch the assertion error and pass it onto Mocha yourself. You do this by wrapping your assertion callback in a try/catch block and pass any exceptions to the done handler.
it('should not fail', function (done) { // Pass reference here!
i_swallow_errors(function (err, result) {
try { // boilerplate to be able to get the assert failures
assert.ok(true);
assert.equal(result, 'bar');
done();
} catch (error) {
done(error);
}
});
});
This boilerplate can of course be extracted into some utility function to make the test a little more pleasing to the eye:
it('should not fail', function (done) { // Pass reference here!
i_swallow_errors(handleError(done, function (err, result) {
assert.equal(result, 'bar');
}));
});
// reusable boilerplate to be able to get the assert failures
function handleError(done, fn) {
try {
fn();
done();
} catch (error) {
done(error);
}
}
Speeding up network tests
Other than that I suggest you pick up the advice on starting to use test stubs for network calls to make tests pass without having to rely on a functioning network. Using Mocha, Chai and Sinon the tests might look something like this
describe('api tests normally involving network calls', function() {
beforeEach: function () {
this.xhr = sinon.useFakeXMLHttpRequest();
var requests = this.requests = [];
this.xhr.onCreate = function (xhr) {
requests.push(xhr);
};
},
afterEach: function () {
this.xhr.restore();
}
it("should fetch comments from server", function () {
var callback = sinon.spy();
myLib.getCommentsFor("/some/article", callback);
assertEquals(1, this.requests.length);
this.requests[0].respond(200, { "Content-Type": "application/json" },
'[{ "id": 12, "comment": "Hey there" }]');
expect(callback.calledWith([{ id: 12, comment: "Hey there" }])).to.be.true;
});
});
See Sinon's nise docs for more info.
If you are using arrow functions:
it('should do something', async () => {
// do your testing
}).timeout(15000)
A little late but someone can use this in future...You can increase your test timeout by updating scripts in your package.json with the following:
"scripts": {
"test": "test --timeout 10000" //Adjust to a value you need
}
Run your tests using the command test
For me the problem was actually the describe function,
which when provided an arrow function, causes mocha to miss the
timeout, and behave not consistently. (Using ES6)
since no promise was rejected I was getting this error all the time for different tests that were failing inside the describe block
so this how it looks when not working properly:
describe('test', () => {
assert(...)
})
and this works using the anonymous function
describe('test', function() {
assert(...)
})
Hope it helps someone, my configuration for the above:
(nodejs: 8.4.0, npm: 5.3.0, mocha: 3.3.0)
My issue was not sending the response back, so it was hanging. If you are using express make sure that res.send(data), res.json(data) or whatever the api method you wanna use is executed for the route you are testing.
Make sure to resolve/reject the promises used in the test cases, be it spies or stubs make sure they resolve/reject.