I'm currently working on a node.js project. We use Q library for promises (https://github.com/kriskowal/q).
We are using mocha for tests and code-coverage provided with grunt tasks (https://github.com/pghalliday/grunt-mocha-test) which uses blanket.js for coverage and travis-cov reporter for asserting the coverage threshold.
Unfortunately the solution does not provide branches coverage for promises.
I have tried Intern (http://theintern.io/), however basic example I wrote does not show correct branch coverage too.
Can you recommend a solution that would provide correct coverage for Q promises and works with grunt and node seamlessly?
Well, checking promises for coverage should not be too hard because JavaScript has absolutely sick aspect oriented programming abilities. While automated tools are offtopic here, let's go through what branches a promise has:
First, the resolver function for the promise (either a promise constructor or a deferred) have their own branches.
Each .then clause has 3 branches (one for success, one for failure, one for progress), if you want loop coverage too, you'd want each progress event when progress is attached to fire zero times, once and multiple times - although let's avoid progress for this question and in general since it's being deprecated and replaced in Q.
The .done, .catch etc... are all private cases of .then. Aggregation methods like .all are private cases of the promise constructor since they create a new aggregate promise and their .then needs to be checked just as well.
So, let's see how we can do this, first - note that Q uses Promise.prototype.then internally for aggregation methods, so we can override it:
Promise.prototype._then = Promise.prototype.then;
var branches = 0;
Promise.prototype.then = function(fulfilled,rejected,progressed){
branches += (fulfilled.call) +(rejected.call) + (progressed.call);
var nFulfilled = function(){ branches--;return fulfilled.apply(this,arguments); };
var nRejected = function(){ branches--; return rejected.apply(this,arguments); };
//progression may happen more than once
var nProgressed = function(){
if(!nProgress.happened) {
branches--;
nProgress.happened = true;
}
return progressed.apply(this,arguments);
};
nProgressed.happened = false;
return this._then((fulfilled.call) ? nFulfilled : undefined,
(rejected.call) ? nRejected : undefined,
(progressed.call) ? nProgressed : undefined);
};
What we're doing here, is hooking on every .then and keeping track of handlers attached and handlers being called.
The branches variable will contain the number of code paths created with promises, and not actually executed. You can use more sophisticated logic here (for example - count rather than subtract) and I'd like to see someone pick up the glove and make a git repo from this :)
Related
I got a lot of callbacks in this application that ended up as a good example of a 'callback-hell'. Methods with 200 or 300 lines and 10, 15 nested callbacks each.
Then, as a good practice, I detached most of these callbacks, named then and created more organized 'then-chains' with Bluebird promises.
I don't know exactly why, since all tests and methods I have implemented are very similar to each other, two tests cases of the same method failed.
It was using Sinon to check whether or not the next function and a DAO method (that makes a Mongo query) were being called.
I then realized that what changed in this method was that I transfered some logic that was running in the beginning of that method to inside a promise.
The call to 'next' was not getting evaluated to true because the test didn't wait for the promise to complete and returned wrongly.
This is the part of the test that showed a bad behavior:
var nextSpy = sinon.spy();
var methodSpy = sinon.spy(my_controller.options.dao, "findAsync");
my_controller.sendMail(req, res, nextSpy);
expect(methodSpy.called).to.be.true;
expect(sendMailSpy.called).to.be.false;
I have changed the controller method, but basically, i had the logic inside of a promise. puting it out solves the problem.
I'm working through a pluralsight course on gulp. John Papa is demonstrating how to inject a function that deletes existing css files, into the routine that compiles the new ones.
The callback on the del function is not firing. The del function is running, file are deleted, I see no error messages. If I call the callback manually it executes, so looks like the function is in tact. So I am wondering what would cause del not to want to execute the callback.
delete routine:
function clean(path, done) {
log('cleaning ' + path);
del(path, done); // problem call
}
The 'done' function is not firing, but it does if I change the code to this:
function clean(path, done) {
log('cleaning ' + path);
del(path);
done();
}
Which, of course, defeats the intended purpose of waiting until del is done before continuing on.
Any ideas at to what's going on would be appreciated.
for reference (in case relevant):
compile css function:
gulp.task('styles', ['clean-styles'], function(){
log('compiling less');
return gulp
.src(config.less)
.pipe($.less())
.pipe($.autoprefixer({browsers:['last 2 versions', '> 5%']}))
.pipe(gulp.dest(config.temp));
});
injected clean function:
gulp.task('clean-styles', function(done){
var files = config.temp + '/**/*.css';
clean(files, done);
});
UPDATE
If anyone else runs into this, re-watched the training video and it was using v1.1 of del. I checked and I was using 2.x. After installing v 1.1 all works.
del isn't a Node's command, it's probably this npm package. If that's the case it doesn't receive a callback as second parameter, instead it returns a promise and you should call .then(done) to get it called after the del finishes.
Update
A better solution is to embrace the Gulp's promise nature:
Change your clean function to:
function clean(path) {
return del(path); // returns a promise
}
And your clean-styles task to:
gulp.task('clean-styles', function(){
var files = config.temp + '/**/*.css';
return clean(files);
});
As of version 2.0, del's API changed to use promises.
Thus to specify callback you should use .then():
del('unicorn.png').then(callback);
In case you need to call it from a gulp task - just return a promise from the task:
gulp.task('clean', function () {
return del('unicorn.png');
});
Checking the docs for the del package it looks like you're getting mixed up between node's standard callback mechanism and del's, which is using a promise.
You'll want to use the promise API, with .then(done) in order to execute the callback parameter.
Node and javascript in general is currently in a bit of a state of flux for design patterns to handle async code, with most of the browser community and standards folks leaning towards promises, whereas the Node community tends towards the callback style and a library such as async.
With ES6 standardizing promises, I suspect we're going to see more of these kinds of incompatibilities in node as the folks who are passionate about that API start incorporating into node code more and more.
(Using Sails.js)
I am testing webworker-threads ( https://www.npmjs.com/package/webworker-threads ) for long running processes on Node and the following example looks good:
var Worker = require('webworker-threads').Worker;
var fibo = new Worker(function() {
function fibo (n) {
return n > 1 ? fibo(n - 1) + fibo(n - 2) : 1;
}
this.onmessage = function (event) {
try{
postMessage(fibo(event.data));
}catch (e){
console.log(e);
}
}
});
fibo.onmessage = function (event) {
//my return callback
};
fibo.postMessage(40);
But as soon as I add any code to query Mongodb, it throws an exception:
(not using the Sails model in the query, just to make sure the code could run on its own -- db has no password)
var Worker = require('webworker-threads').Worker;
var fibo = new Worker(function() {
function fibo (n) {
return n > 1 ? fibo(n - 1) + fibo(n - 2) : 1;
}
// MY DB TEST -- THIS WORKS FINE OUTSIDE THE WORKER
function callDb(event){
var db = require('monk')('localhost/mydb');
var users = db.get('users');
users.find({ "firstName" : "John"}, function (err, docs){
console.log(("serviceSuccess"));
return fibo(event.data);
});
}
this.onmessage = function (event) {
try{
postMessage(callDb(event.data)); // calling db function now
}catch (e){
console.log(e);
}
}
});
fibo.onmessage = function (event) {
//my return callback
};
fibo.postMessage(40);
Since the DB code works perfectly fine outside the Worker, I think it has something to do with the require. I've tried something that also works outside the Worker, like
var moment = require("moment");
var deadline = moment().add(30, "s");
And the code also throws an exception. Unfortunately, console.log only shows this for all types of errors:
{Object}
{/Object}
So, the questions are: is there any restriction or guideline for using require inside a Worker? What could I be doing wrong here?
UPDATE
it seems Threads will not allow external modules
https://github.com/xk/node-threads-a-gogo/issues/22
TL:DR I think that if you need to require, you should use a node's
cluster or child process. If you want to offload some cpu busy work,
you should use tagg and the load function to grab any helpers you
need.
Upon reading this thread, I see that this question is similar to this one:
Load Nodejs Module into A Web Worker
To which Audreyt, the webworker-threads author answered:
author of webworker-threads here. Thank you for using the module!
There is a default native_fs_ object with the readFileSync you can use
to read files.
Beyond that, I've mostly relied on onejs to compile all required
modules in package.json into a single JS file for importScripts to
use, just like one would do when deploying to a client-side web worker
environment. (There are also many alternatives to onejs -- browserify,
etc.)
Hope this helps!
So it seems importScripts is the way to go. But at this point, it might be too hacky for what I want to do, so probably KUE is a more mature solution.
I'm a collaborator on the node-webworker-threads project.
You can't require in node-webworker-threads
You are correct in your update: node-webworker-threads does not (currently) support requireing external modules.
It has limited support for some of the built-ins, including file system calls and a version of console.log. As you've found, the version of console.log implemented in node-webworker-threads is not identical to the built-in console.log in Node.js; it does not, for example, automatically make nice string representations of the components of an Object.
In some cases you can use external modules, as outlined by audreyt in her response. Clearly this is not ideal, and I view the incomplete require as the primary "dealbreaker" of node-webworker-threads. I'm hoping to work on it this summer.
When to use node-webworker-threads
node-webworker-threads allows you to code against the WebWorker API and run the same code in the client (browser) and the server (Node.js). This is why you would use node-webworker-threads over node-threads-a-gogo.
node-webworker-threads is great if you want the most lightweight possible JavaScript-based workers, to do something CPU-bound. Examples: prime numbers, Fibonacci, a Monte Carlo simulation, offloading built-in but potentially-expensive operations like regular expression matching.
When not to use node-webworker-threads
node-webworker-threads emphasizes portability over convenience. For a Node.js-only solution, this means that node-webworker-threads is not the way to go.
If you're willing to compromise on full-stack portability, there are two ways to go: speed and convenience.
For speed, try a C++ add-on. Use NaN. I recommend Scott Frees's C++ and Node.js Integration book to learn how to do this, it'll save you a lot of time. You'll pay for it in needing to brush up on your C++ skills, and if you want to work with MongoDB then this probably isn't a good idea.
For convenience, use a Child Process-based worker pool like fork-pool. In this case, each worker is a full-fledged Node.js instance. You can then require to your heart's content. You'll pay for it in a larger application footprint and in higher communication costs compared to node-webworker-threads or a C++ add-on.
I'm trying to understand how Sinon is used properly in a node project. I have gone through examples, and the docs, but I'm still not getting it. I have setup a directory with the following structure to try and work through the various Sinon features and understand where they fit in
|--lib
|--index.js
|--test
|--test.js
index.js is
var myFuncs = {};
myFuncs.func1 = function () {
myFuncs.func2();
return 200;
};
myFuncs.func2 = function(data) {
};
module.exports = myFuncs;
test.js begins with the following
var assert = require('assert');
var sinon = require('sinon');
var myFuncs = require('../lib/index.js');
var spyFunc1 = sinon.spy(myFuncs.func1);
var spyFunc2 = sinon.spy(myFuncs.func2);
Admittedly this is very contrived, but as it stands I would want to test that any call to func1 causes func2 to be called, so I'd use
describe('Function 2', function(){
it('should be called by Function 1', function(){
myFuncs.func1();
assert(spyFunc2.calledOnce);
});
});
I would also want to test that func1 will return 200 so I could use
describe('Function 1', function(){
it('should return 200', function(){
assert.equal(myFuncs.func1(), 200);
});
});
but I have also seen examples where stubs are used in this sort of instance, such as
describe('Function 1', function(){
it('should return 200', function(){
var test = sinon.stub().returns(200);
assert.equal(myFuncs.func1(test), 200);
});
});
How are these different? What does the stub give that a simple assertion test doesn't?
What I am having the most trouble getting my head around is how these simple testing approaches would evolve once my program gets more complex. Say I start using mysql and add a new function
myFuncs.func3 = function(data, callback) {
connection.query('SELECT name FROM users WHERE name IN (?)', [data], function(err, rows) {
if (err) throw err;
names = _.pluck(rows, 'name');
return callback(null, names);
});
};
I know when it comes to databases some advise having a test db for this purpose, but my end-goal might be a db with many tables, and it could be messy to duplicate this for testing. I have seen references to mocking a db with sinon, and tried following this answer but I can't figure out what's the best approach.
You have asked so many different questions on one post... I will try to sort out.
Testing myFuncs with two functions.
Sinon is a mocking library with wide features. "Mocking" means you are supposed to replace some part of what is going to be tested with mocks or stubs. There is a good article among Sinon documentation which describes the difference well.
When you created a spy in this case...
var spyFunc1 = sinon.spy(myFuncs.func1);
var spyFunc2 = sinon.spy(myFuncs.func2);
...you've just created a watcher. myFuncs.func1 and myFuncs.func2 will be substituted with a spy-function, but it will be used to record the calling arguments and call real function after that. This is a possible scenario, but mind that all possibly complicated logic of myFuncs.func1/func2 will run after being called in the test (ex.: database query).
2.1. The describe('Function 1', ...) test suite looks too contrived to me.
It is not obvious which question you actually mean.
A function that returns a constant value is not a real life example.. In most cases there will be some parameters and the function under test will implement some algorithm of transmuting the input arguments. So in your test you're going to implement the same algorithm partly to check that the function works correctly. That is where TDD comes in place, which actually supposes you start implementation from a test and take parts of unit test code to implement the method being tested.
2.2. Stub. The second version of unit test looks useless in the given example. func1 is not accepting any parameter.
var test = sinon.stub().returns(200);
assert.equal(myFuncs.func1(test), 200);
Even if you replace the return part with 100 the test will run successfully.
The one that makes sense is, for example, replacing func2 with a stub to avoid heavy calculation/remote request (DB query, http or other API request) being launched in a test.
myFuncs.func2 = sinon.spy();
assert.equal(myFuncs.func1(test), 200);
assert(myFuncs.func2.calledOnce);
The basic rule for unit testing is that unit test should be kept as simple as possible, providing check for as smallest possible fragment of code. In this test func1 is being tested so we can neglect the logic of func2. Which should be tested in another unit test.
Mind that doing the following attempt is useless:
myFuncs.func1 = sinon.stub().returns(200);
assert.equal(myFuncs.func1(test), 200);
Because in this case you masked the real func1 logic with a stub and you're actually testing sinon.stub().return(). Believe me it works well! :D
Mocking database queries.
Mocking a database has always been a hurdle. I could provide some advises.
3.1. Have well fragmented environment. Even for a small project there better exist a development, stage and production completely independent environments. Including databases. Which means you have an automated way of DB creation: scripts or ORM.
In this scenario you will be easily maintaining test DB in your test engine using before()/beforeEach() to have a clean structure for your tests.
3.2. Have well fragmented code. There'd better exist several layers. The lowest (DAL) should be separated from business logic. In this case you will write code for business class, simply mocking DAL. To test DAL you can have the approach you mentioned (sinon.mock the whole module) or some specific libraries (ex.: replace db engine with SQLite for tests as described here)
Conclusion. "how these simple testing approaches would evolve once my program gets more complex".
It is hard to maintain unit tests unless you develop your application with tests in mind and, thus, well fragmented. Stick to the main rule - keep each unit test as small as possible. Otherwise you're right, it will eventually get messy. Because the constantly evolving logic of your application will be involved in your test code.
I installed sqlite3 with npm
npm install sqlite3 --save
I have written some basic functions that I would like to be performed synchronously rather then with async.
for instance I would like to get column names in one function and the row count in another.
I would like to simply return these values. currently I am using a callback like so
d.cinfo = function(table, callback){
var o = {};
db.each("PRAGMA table_info(" + table + ")", function(err, col){
o[col.name] = col.type;
}, function(){
if(typeof callback == 'function') callback(o);
});
}
d is an object which is later exposed
however Id like to return the values
d.cinfo = function(table, callback){
var o = {};
db.each("PRAGMA table_info(" + table + ")", function(err, col){
o[col.name] = col.type;
}, function(){
return o;
});
}
is there a way I can achieve this. I found documentation saying it was possible but then I learned that it was outdated and im not sure its meant for the same api
I have implemented bluebird however Promise.promisifyAll(middleware) returns and error "Object # has no method .then() anyone know what im doing wrong
I've come across a problem like this and many others while using Express and it really drives me crazy where I just can't avoid callback hells.
Some of the possible solutions can be:
Using the same old event driven style. Instead of calling nested functions, emit events for them. But I'm assuming you wouldn't like that very much, but that atleast gets rid of the nested callbacks.
There's a npm module called bluebird that allows you to extend existing modules with promises. (Using promisifyAll). So promises will still trim down the code and you'll have much cleaner code than you had before using promises.
If you don't mind switching frameworks, you can try KoaJS that allows you to yield(fun stuff from ECS 6) functions using generator functions(another cool feature). You can have more reading about this here.
So the above code can be re-written like:
var rows = yield db.each() //or something like that
this is an amazing feature but the drawback is that you have to use unstable versions of node (0.11.x). We also faced some issues setting up https using 0.11.x and had to downgrade to 0.10.x (we're not using koa either)
If there are only a few places where you need to do such kind of stuff, I would recommend using bluebird, but if you're tired of having lots of callbacks in your code, better go for Koa, although I've already mentioned you the tradeoffs.