I have minimal experience with Node.js projects and have inherited one which uses Gulp to build and distribute itself.
The project has several disparate task files, and a gulpfile.js which references them all. For brevity, I will use the smallest function as an example.
//gulpfile.js
...
gulp.task('csscompile', function() {
gulp.src('./src/ops/gulp/csscompile.js')
});
...
//csscompile.js
...
module.exports = function (gulp) {
gulp.task('csscompile', function (done) {
let paths = [
'src/app/**/*.scss',
'node_modules/angular-material/angular-material.scss',
];
gulp.src(paths)
.pipe(sass())
.pipe(concat('app.css'))
.pipe(gulp.dest('build'))
.done();
});
};
Running $ gulp csscompile from the command line always completes the entire function in about 10ms, and I get output like:
[10:44:43] The following tasks did not complete: csscompile
[10:44:43] Did you forget to signal async completion?
Obviously there is an issue with waiting for the functions to complete. After searching around I have tried every imaginable combination of using async flags, using function (done), and more, to no avail. Even putting the csscompile function directly in the csscompile task does not work.
I assume using gulp.src() is part of the issue, I don't know whether those functions are inlined or what. I also want to avoid having to turn every one of these functions into a Promise.
Does anyone have any recommendations of changes I can make?
According to the docs:
When you see the "Did you forget to signal async completion?" warning, none of the techniques mentioned above were used. You'll need to use the error-first callback or return a stream, promise, event emitter, child process, or observable to resolve the issue.
So you need to return the stream.
This should work:
module.exports = function (gulp) {
gulp.task('csscompile', function () {
let paths = [
'src/app/**/*.scss',
'node_modules/angular-material/angular-material.scss',
];
return gulp.src(paths)
.pipe(sass())
.pipe(concat('app.css'))
.pipe(gulp.dest('build'));
});
Related
In my Cordova project, I have a hook which does RequireJS optimization (r.js) on after_prepare. That optimization is inherently asynchronous, so my hook code returns before all optimization is fully finished.
For example, this causes issues when running cordova run browser: On the first page load, optimization has not finished yet and the site looks broken.
Is there a way to make the Cordovoa build process to block until a certain hook fires a callback? Or can the optimizer be run in a blocking/sync way?
An alternative I could think of is using a different process for optimization and busy-wait in the main for it to finish, but that looks like an overkill and bad practice to me.
You can use the built-in promise module to block Cordova from proceeding until the hook has resolved.
Something along these lines:
#!/usr/bin/env node
var deferral;
function doSomethingAsync(){
somethingAync
.success(function(){
deferral.resolve();
})
.fail(function(err){
deferral.reject(err);
});
}
module.exports = function(ctx) {
deferral = ctx.requireCordovaModule('q').defer();
doSomethingAsync();
return deferral.promise;
};
You don't need to call context, you can simply return a Promise from within the module.exports function
module.exports = context => {
return new Promise(resolve => {
setTimeout(() => resolve(), 1000);
});
};
I tested and it works. The problem arises because in Cordova versions => 9 you cannot use context.requireCordovaModule('q')
If you don't want to use Promise, just do
module.exports = context => {
var deferral = require('q').defer();
doSomethingAsync(() => {
deferral.resolve();
});
return deferral.promise;
};
I understand what a callback is and what asynchronous means, what I don't get is how to run asynchronous functions in node.
For example, how is this
var action = (function(data,callback) {
result = data+1;
callback(result);
});
http.createServer(function (req, res) {
action(5, function(r){
res.end(r.toString());
});
}).listen(80);
different from this
var action = (function(data) {
result = data+1;
return result;
});
http.createServer(function (req, res) {
var r = action(5);
res.end(r.toString());
}).listen(80);
?
I guess in the first example I'm doing it asynchronously, yet I don't know how Node knows when to do it sync or async... is it a matter of the return? or the fact that in the sync mode we're doing var x = func(data);?
And also: when to use sync or async? Because obviously you don't want to use it when adding +1... is it OK to use async just when performing IO tasks, such as reading from DB?
For example, I'm using the library crypto to encrypt a short string (50 chars at most), is this case a good example where I should already be using async?
I guess in the first example I'm doing it asynchronously...
Your first example isn't async :) Merely passing a callback and calling it when you're done doesn't make a function asynchronous.
Asynchronous means that, basically, you're telling Node: "here, do this for me, and let me know when you're done while I continue doing other stuff".
Your example is not handing anything to Node for future completion. It's doing a calculation and calling the callback immediately after that. That's functionally the same as your second example, where you return the result of the calculation.
However, you can change your first example to something that is asynchronous:
var action = (function(data,callback) {
setTimeout(function() {
result = data + 1;
callback(result);
}, 1000);
});
Here, you're telling Node to delay calling the callback for one second, by using setTimeout. In the mean time, Node won't get stuck waiting for a second; it will happily accept more HTTP requests, and each one will be delayed one second before the response is sent.
When to use sync or async?
Asynchronous code is "viral": if you rely on functions that are async, your own code that uses those functions will also have to be async (generally by accepting a callback, or using another mechanism to deal with asynchronicity, like promises).
For example, I'm using the library crypto to encrypt a short string (50 chars at most), is this case a good example where I should already be using async?
This depends on which function you're using. AFAIK, most encryption functions in crypto aren't asynchronous, so you can't "make" them asynchronous yourself.
Both examples will work synchronous. Simple async operations are setTimout and setInterval.
Node actually doesn't care what code are you running. You can block or not (blocking/non-blocking).
In other words - you have event loop. If your process is async he will pass the program control to the event loop, so it can execute any other action node needs to be done. If not - he wont.
if you want a function to work asynchronously, you can do that using promises, look at the code below :
function is_asynch(){
return new Promise((resolve,reject)=>{
resolve( here_your_synch_function() )
})
}
I've got a NodeJS app i'm building (using Sails, but i guess that's irrelevant).
In my action, i have a number of requests to other services, datasources etc that i need to load up. However, because of the huge dependency on callbacks, my code is still executing long after the action has returned the HTML.
I must be missing something silly (or not quite getting the whole async thing) but how on earth do i stop my action from finishing until i have all my data ready to render the view?!
Cheers
I'd recommend getting very intimate with the async library
The docs are pretty good with that link above, but it basically boils down to a bunch of very handy calls like:
async.parallel([
function(){ ... },
function(){ ... }
], callback);
async.series([
function(){ ... },
function(){ ... }
]);
Node is inherently async, you need to learn to love it.
It's hard to tell exactly what the problem is but here is a guess. Assuming you have only one external call your code should look like this:
exports.myController = function(req, res) {
longExternalCallOne(someparams, function(result) {
// you must render your view inside the callback
res.render('someview', {data: result});
});
// do not render here as you don't have the result yet.
}
If you have more than two external calls your code will looks like this:
exports.myController = function(req, res) {
longExternalCallOne(someparams, function(result1) {
longExternalCallTwo(someparams, function(result2) {
// you must render your view inside the most inner callback
data = {some combination of result1 and result2};
res.render('someview', {data: data });
});
// do not render here since you don't have result2 yet
});
// do not render here either as you don't have neither result1 nor result2 yet.
}
As you can see, once you have more than one long running async call things start to get tricky. The code above is just for illustration purposes. If your second callback depends on the first one then you need something like it, but if longExternalCallOne and longExternalTwo are independent of each other you should be using a library like async to help parallelize the requests https://github.com/caolan/async
You cannot stop your code. All you can do is check in all callbacks if everything is completed. If yes, go on with your code. If no, wait for the next callback and check again.
You should not stop your code, but rather render your view in your other resources callback, so you wait for your resource to be reached before rendering. That's the common pattern in node.js.
If you have to wait for several callbacks to be called, you can check manually each time one is called if the others have been called too (with simple bool for example), and call your render function if yes. Or you can use async or other cool libraries which will make the task easier. Promises (with the bluebird library) could be an option too.
I am guessing here, since there is no code example, but you might be running into something like this:
// let's say you have a function, you pass it an argument and callback
function myFunction(arg, callback) {
// now you do something asynchronous with the argument
doSomethingAsyncWithArg(arg, function() {
// now you've got your arg formatted or whatever, render result
res.render('someView', {arg: arg});
// now do the callback
callback();
// but you also have stuff here!
doSomethingElse();
});
});
So, after you render, your code keeps running. How to prevent it? return from there.
return callback();
Now your inner function will stop processing after it calls callback.
I understand that node.js has a lot of functions that runs async, like the I/O functions.
I am trying to run two modules that I created in a async way, these modules only use the CPU, not an I/O request.
For example, I have something like this: I call f2() first, but I want it so if f1() ends first it won't wait until f2() has finished; is that possible?
This is only an example, but I will use this calling another js file.
I tried to write this using the process.nextTick(), but I don't know if that's the real solution.
function f1(){
for (i=0;i< 100;i++){
}
console.log("f1")
}
function f2(){
for (i=0;i< 100000000000;i++){
}
console.log("f2")
}
f2();
f1();
There is an async module for doing just this
look at the parallel example in the following post node.js async libs
async.parallel([
function(){ ... },
function(){ ... }
], callback);
the wiki for async.js https://github.com/caolan/async
i try to understand and use async library in node. What i am really not understand, how async.parallel function worked. The name parallel it seems for like multithreading, consider the following sample:
async.parallel([
function(callback){
setTimeout(function(){
console.log('1');
callback(null, 'one');
}, 200);
},
function(callback){
setTimeout(function(){
console.log('2');
callback(null, 'two');
}, 100);
}
],
// optional callback
function(err, results){
if(err){
console.log('Error');
} else {
console.log(results);
}
// the results array will equal ['one','two'] even though
// the second function had a shorter timeout.
});
i have got the result
[ 'one', 'two' ]
Do async.parallel execute on multithread? If not, what the name parallel express?
The placement of results within the array is gauranteed by your placement of those functions within the array you pass to parallel. Notice that functions you pass to parallel have to take the same arguments. Picture these functions being placed in a wrapper and then called like this
someArrayOfFunctions[index](arguments);
Using this method, async gaurantees that independent of when functions within parallel finishes, the results are placed in the array in the order expected, through use of callbacks for populating results, based on indices. The specific implementation does not matter, the fact is no matter when you finish, the results of respective runs will end up placed based on where their function was in the array, not based on timing.
As per your second question, whether async.parallel is truly parallel. It is not. Refer to the following example:
var async = require('async');
async.parallel([
function(callback){
while(true);//blocking!
},
function(callback){
console.log('Not Blocked');//Never get's called
}
]);
The second function will never get called. The parallel that async provides helps out, for exactly the reason you are confused about your example. The problem with asynchronous code is that, sometimes we have some series of callbacks for things that are actually parallel in node(disk I/O, network I/O, etc...) that need to complete, but that will end up completing at unpredictable intervals. Say for example we have configuration data to collect from multiple sources, and no sync methods are supplied. We don't want to run these serially, because this slows things down considerably, but we still want to collect the results in order. This is the prime example use for async.parallel. But, no, async.parallel cannot make synchronous code execute asynchronously, as is shown by this blocking example.
True parallelism in node comes within the V8 backend. The only way to provide this parallelism would be to release an alternate version of node or by developing native extensions.