I understand that node.js has a lot of functions that runs async, like the I/O functions.
I am trying to run two modules that I created in a async way, these modules only use the CPU, not an I/O request.
For example, I have something like this: I call f2() first, but I want it so if f1() ends first it won't wait until f2() has finished; is that possible?
This is only an example, but I will use this calling another js file.
I tried to write this using the process.nextTick(), but I don't know if that's the real solution.
function f1(){
for (i=0;i< 100;i++){
}
console.log("f1")
}
function f2(){
for (i=0;i< 100000000000;i++){
}
console.log("f2")
}
f2();
f1();
There is an async module for doing just this
look at the parallel example in the following post node.js async libs
async.parallel([
function(){ ... },
function(){ ... }
], callback);
the wiki for async.js https://github.com/caolan/async
Related
I have minimal experience with Node.js projects and have inherited one which uses Gulp to build and distribute itself.
The project has several disparate task files, and a gulpfile.js which references them all. For brevity, I will use the smallest function as an example.
//gulpfile.js
...
gulp.task('csscompile', function() {
gulp.src('./src/ops/gulp/csscompile.js')
});
...
//csscompile.js
...
module.exports = function (gulp) {
gulp.task('csscompile', function (done) {
let paths = [
'src/app/**/*.scss',
'node_modules/angular-material/angular-material.scss',
];
gulp.src(paths)
.pipe(sass())
.pipe(concat('app.css'))
.pipe(gulp.dest('build'))
.done();
});
};
Running $ gulp csscompile from the command line always completes the entire function in about 10ms, and I get output like:
[10:44:43] The following tasks did not complete: csscompile
[10:44:43] Did you forget to signal async completion?
Obviously there is an issue with waiting for the functions to complete. After searching around I have tried every imaginable combination of using async flags, using function (done), and more, to no avail. Even putting the csscompile function directly in the csscompile task does not work.
I assume using gulp.src() is part of the issue, I don't know whether those functions are inlined or what. I also want to avoid having to turn every one of these functions into a Promise.
Does anyone have any recommendations of changes I can make?
According to the docs:
When you see the "Did you forget to signal async completion?" warning, none of the techniques mentioned above were used. You'll need to use the error-first callback or return a stream, promise, event emitter, child process, or observable to resolve the issue.
So you need to return the stream.
This should work:
module.exports = function (gulp) {
gulp.task('csscompile', function () {
let paths = [
'src/app/**/*.scss',
'node_modules/angular-material/angular-material.scss',
];
return gulp.src(paths)
.pipe(sass())
.pipe(concat('app.css'))
.pipe(gulp.dest('build'));
});
I wrote the following function that takes a callback. I always thought that the content of a callback might be executed later. In this case it doesn't...
doesSomething(function(){
console.log("1");
var i = 0;
while (i < 10000)
{
console.log("hello");
i = i + 1;
}
});
console.log("2");
console.log("3");
Whatever I do, "2" and "3" always comes after "1" and a thousand of "hello".
Like this:
1
hello
hello
...
hello
hello
2
3
What I thought it would do:
2
3
1
hello
hello
...
hello
hello
Even if this behaviour makes my life simpler I don't really understand why the execution is procedural.
Do you think that in some case it might go reverse ?
It all depends upon how doSomething() calls its callback. If it calls the callback synchronously (e.g. before it returns), then everything in that function will execute before doSomething() returns. If it calls it asynchronously (sometime after it returns), then you will get a different order.
So, the order is determined by the code that you do not show us in doSomething().
Based on the order you observe, doSomething() must be calling its callback synchronously and thus it will execute in order just like any other synchronous function call.
For example, here are two scenarios:
function doSomething(callback) {
callback();
}
This calls the callback passed to it synchronously and thus it will execute before doSomething() returns and thus it will execute before code that follows.
Whereas something like this;
function doSomething(callback) {
fs.writeFile('foo.txt', callback);
}
or:
function doSomething(callback) {
setTimeout(callback, 50);
}
Will execute the callback asynchronously sometime later after the function has already returned and you will see a different execution order with your console.log() statements.
From the output that you mention, I can say that there is nothing asynchronous happening in your code inside doesSomething and is probably looking like this:
function doesSomething(callback) {
<maybe some synchronous operations, e.g. no ajax or filesystem calls>
callback();
}
, so the order of the calling functions will be always the same, like you posted above:
doesSomething(callback)
console.log(2)
console.log(3)
If you want the order you mention you should call them like this:
console.log(2)
console.log(3)
doesSomething(callback)
This does not have to do something with node.js in particular. It is the way methods are executed in JavaScript, which is synchronous since it is single threaded.
When you call the method doesSomething(callback) it immediately executes it and nothing else until it completes
javascript is a single threaded language, but not the node.js runtime or the browser. There are certain functions provided by node.js or the browser that would trigger for a function to be assigned to a task queue to be processed in a separate thread
lets take your example and make dosomething async
this is your synchronous code that would print, 1 hello...,2,3
function doesSomething(callback) {
callback();
}
doesSomething(function(){
console.log("1");
var i = 0;
while (i < 3)
{
console.log("hello");
i = i + 1;
}
});
console.log("2");
console.log("3");
and this is the async code, that prints out 3, 2, 1, hello ...
function doesSomething(callback) {
setTimeout(callback, 0);
}
doesSomething(function(){
console.log("1");
var i = 0;
while (i < 3)
{
console.log("hello");
i = i + 1;
}
});
console.log("2");
console.log("3");
to explain why the second code is async, you have to understand that setTimeout is not part of javascript, its an api provided by node.js and the browsers. setTimeout puts the callback function into a queue to be processed. At this time a separate thread runs the setTimeout and when the timer ends it puts the callback into a callback queue, and when the call stack is clear, whatever is in the callback queue will be moved to the call stack and processed.
in node you can use process.nextTick(yourFunction); to make it async, this is just another function that nodejs provides you which is better than using setTimeout (which I wont get into here) you can checkout https://howtonode.org/understanding-process-next-tick to understand more about process.nextTick
for more info check out this video https://youtu.be/8aGhZQkoFbQ?t=19m25s
I understand what a callback is and what asynchronous means, what I don't get is how to run asynchronous functions in node.
For example, how is this
var action = (function(data,callback) {
result = data+1;
callback(result);
});
http.createServer(function (req, res) {
action(5, function(r){
res.end(r.toString());
});
}).listen(80);
different from this
var action = (function(data) {
result = data+1;
return result;
});
http.createServer(function (req, res) {
var r = action(5);
res.end(r.toString());
}).listen(80);
?
I guess in the first example I'm doing it asynchronously, yet I don't know how Node knows when to do it sync or async... is it a matter of the return? or the fact that in the sync mode we're doing var x = func(data);?
And also: when to use sync or async? Because obviously you don't want to use it when adding +1... is it OK to use async just when performing IO tasks, such as reading from DB?
For example, I'm using the library crypto to encrypt a short string (50 chars at most), is this case a good example where I should already be using async?
I guess in the first example I'm doing it asynchronously...
Your first example isn't async :) Merely passing a callback and calling it when you're done doesn't make a function asynchronous.
Asynchronous means that, basically, you're telling Node: "here, do this for me, and let me know when you're done while I continue doing other stuff".
Your example is not handing anything to Node for future completion. It's doing a calculation and calling the callback immediately after that. That's functionally the same as your second example, where you return the result of the calculation.
However, you can change your first example to something that is asynchronous:
var action = (function(data,callback) {
setTimeout(function() {
result = data + 1;
callback(result);
}, 1000);
});
Here, you're telling Node to delay calling the callback for one second, by using setTimeout. In the mean time, Node won't get stuck waiting for a second; it will happily accept more HTTP requests, and each one will be delayed one second before the response is sent.
When to use sync or async?
Asynchronous code is "viral": if you rely on functions that are async, your own code that uses those functions will also have to be async (generally by accepting a callback, or using another mechanism to deal with asynchronicity, like promises).
For example, I'm using the library crypto to encrypt a short string (50 chars at most), is this case a good example where I should already be using async?
This depends on which function you're using. AFAIK, most encryption functions in crypto aren't asynchronous, so you can't "make" them asynchronous yourself.
Both examples will work synchronous. Simple async operations are setTimout and setInterval.
Node actually doesn't care what code are you running. You can block or not (blocking/non-blocking).
In other words - you have event loop. If your process is async he will pass the program control to the event loop, so it can execute any other action node needs to be done. If not - he wont.
if you want a function to work asynchronously, you can do that using promises, look at the code below :
function is_asynch(){
return new Promise((resolve,reject)=>{
resolve( here_your_synch_function() )
})
}
I've got a NodeJS app i'm building (using Sails, but i guess that's irrelevant).
In my action, i have a number of requests to other services, datasources etc that i need to load up. However, because of the huge dependency on callbacks, my code is still executing long after the action has returned the HTML.
I must be missing something silly (or not quite getting the whole async thing) but how on earth do i stop my action from finishing until i have all my data ready to render the view?!
Cheers
I'd recommend getting very intimate with the async library
The docs are pretty good with that link above, but it basically boils down to a bunch of very handy calls like:
async.parallel([
function(){ ... },
function(){ ... }
], callback);
async.series([
function(){ ... },
function(){ ... }
]);
Node is inherently async, you need to learn to love it.
It's hard to tell exactly what the problem is but here is a guess. Assuming you have only one external call your code should look like this:
exports.myController = function(req, res) {
longExternalCallOne(someparams, function(result) {
// you must render your view inside the callback
res.render('someview', {data: result});
});
// do not render here as you don't have the result yet.
}
If you have more than two external calls your code will looks like this:
exports.myController = function(req, res) {
longExternalCallOne(someparams, function(result1) {
longExternalCallTwo(someparams, function(result2) {
// you must render your view inside the most inner callback
data = {some combination of result1 and result2};
res.render('someview', {data: data });
});
// do not render here since you don't have result2 yet
});
// do not render here either as you don't have neither result1 nor result2 yet.
}
As you can see, once you have more than one long running async call things start to get tricky. The code above is just for illustration purposes. If your second callback depends on the first one then you need something like it, but if longExternalCallOne and longExternalTwo are independent of each other you should be using a library like async to help parallelize the requests https://github.com/caolan/async
You cannot stop your code. All you can do is check in all callbacks if everything is completed. If yes, go on with your code. If no, wait for the next callback and check again.
You should not stop your code, but rather render your view in your other resources callback, so you wait for your resource to be reached before rendering. That's the common pattern in node.js.
If you have to wait for several callbacks to be called, you can check manually each time one is called if the others have been called too (with simple bool for example), and call your render function if yes. Or you can use async or other cool libraries which will make the task easier. Promises (with the bluebird library) could be an option too.
I am guessing here, since there is no code example, but you might be running into something like this:
// let's say you have a function, you pass it an argument and callback
function myFunction(arg, callback) {
// now you do something asynchronous with the argument
doSomethingAsyncWithArg(arg, function() {
// now you've got your arg formatted or whatever, render result
res.render('someView', {arg: arg});
// now do the callback
callback();
// but you also have stuff here!
doSomethingElse();
});
});
So, after you render, your code keeps running. How to prevent it? return from there.
return callback();
Now your inner function will stop processing after it calls callback.
I have four functions which are running parallely. If any one function fails in between, how can I stop the execution of other functions. Any help on this will be really helpful.
Except if it's a setTimeout or setInterval, I think you can't. You can, anyway, set checkpoints in the logic of the functions. It's not clean a all, but it can work. For example, checking the other function in the callback:
var control = true;
async1(function(e,r){
if(e) {
control = false;
return callback1(e,r);
};
if(control) callback1(e,r);
});
async2(function(e,r){
if(e) {
control = false;
return callback2(e,r);
};
if(control) callback2(e,r);
});
Althougt to do this, I would go with throrin19 and say that Async it's a nice lib to do this.
But maybe you would like to check co. It could handle your problem better than Async.
You can't stop execution of a function. Functions are executed and return synchronously, so there is technically nothing to stop. But there can be asynchronous tasks, which use underlying libuv capabilites somehow (that is, you can't do anything asynchronous without calling some asynchronous node api or some native module). Functions are only interfaces for such tasks and they don't support canceling tasks, only starting.
So, you can't really cancel async operations, but what you can do is ignore the results of other operations if one fails. Here is how it can be implemented:
var tasks = [], //array of tasks, functions accepting callback(err)
pending = tasks.length, //amount of pending tasks
failed = false;
function done(err) { //callback for each task
if (failed) return;
if (err) {
failed = true;
callback(err); //callback for all tasks
}
if (!--pending) callback(); //all tasks completed
}
tasks.forEach(function(task) {
task(done);
});
I think you should use the Async library. Async contains parallel function that executes functions in parallel (of course). And each function takes a callback (error, result).
If an error occurs, Async will cut all other processes.
Documentation