async.parallel([
function(callback) { //This is the first task, and callback is its callback task
db.save('xxx', 'a', function(err) {
//Now we have saved to the DB, so let's tell async that this task is done
callback();
});
},
function(callback) { //This is the second task, and callback is its callback task
db.save('xxx', 'b', callback); //Since we don't do anything interesting in db.save()'s callback, we might as well just pass in the task callback
}
], function(err) { //This is the final callback
console.log('Both a and b are saved now');
});
Hey I found this code online when I trying to understand async in node.js and my question I have with the code above is in the array of functions in async.parallel, what is the callback parameter that is passed in each function? Where does that callback parameter come from and what is the purpose of that variable? I am sorry if this is a dumb question, but I can't grasp a solid understanding of this callback's...
The contract that async.parallel() defines is that you pass it an array of functions. It will then call each of those functions and pass those function a single argument that is a function reference (I think of it as a completion callback). For async.parallel() to work properly, your function that you pass in the array has to call that function reference when your async operation is done. When it calls that function reference, it can pass an error argument and a result argument. If the error argument is truthy, then it will stop further processing and feed that error back to the calling code.
The function that async passes a reference to is internal to the async library and when you call it, the async library does some housekeeping to keep track of how many asynchronous operations are still going, whether they're all done now, whether further processing should stop due to error, etc...
You can essentially think of it like the async library completion function. When your asynchronous operation is done, you have to call that completion function so that the async library knows that your operation is now done. That's how it is able to manage and keep track of your multiple async operations. If you don't call that completion callback, then the async library never knows when your asynchronous operation is done.
The callback parameter is a function created by async. When you call it in each of the array functions, it signals async that the function completed. In that way, you can control when your function is done - when you call callback.
When callback is called from all the array functions, the final function is called.
Related
Hey there so I'm trying to store the result of a promise in a variable because I have some code that doesn't work inside of the promise.
(I'm pretty new to node.js so a bit of explanation would be awesome!)
And how would I use a async await?
Heres my current code:
const rbx = require("noblox.js")
var myblurb;
rbx.getBlurb(166962499).then(function (blurb) {
myblurb = blurb;
});
console.log(myblurb);
The function that sets your variable will get called after your function completes, not during it. Therefore your variable is still undefined after calling rbx.getBlurb because it returns a promise that will complete later on.
Using async/await:
One way to get what you're after is to use await. This can only be done within a function that is declared as async, and that will mean the function now will also return a Promise. If you're okay with that, then you can do it like this:
async function doStuff() {
var myblurb;
myblurb = await rbx.getBlurb(166962499);
console.log(myblurb);
}
This tells your function to stop and wait until the promise from calling getBlurb has resolved, and assign the resolved value to your local variable.
Note that if there's an error (either a promise rejection or a thrown exception) then your function will pass that along. You could surround it all with a try/catch to handle that if you need.
Operating Within then
If you don't want to make your function return a promise then you could alternately do this:
rbx.getBlurb(166962499).then(function (blurb) {
const myblurb = blurb;
console.log(myblurb);
// myblurb is only useful within this 'then' resolve function
});
It just means that you have to do the rest of your functionality that relies on the myblurb variable within that resolve function of the then.
Note that whoever is calling this code needs to understand that it will be asynchronous, so it should either use a callback function or Promise to give its results back to the caller (if needed)
Your callback function where assignment is happening will be called after promise is called. Console.log will be executed before that. So it will always log undefined.
You have to do console.log inside function of you can also use async await so solve this.
Task: I need to recursively walk through a json object and make certain changes to the keys. I will be handling objects with varying depths, and varying sizes. When the function hits a key whose value is an object, it is called again on that object.
Problem 1.: Doing this as a synchronous function, I noticed that large json objects were returning incomplete. Using the the async library, async.forEach solves the problem of handling long tasks and returning only when finished, but...
Problem 2.: It seems that the async function loses concurrency (?) when it is called recursively (pointed out in the code snippet).
To test this idea, I removed the recursive function call and it worked (but without recursion), and I got a callback. When I add the function call back in, I get TypeError: results is not a function .
This leads me to think that async with callback and recursive functions don't mix in node. Is there a way to achieve both?
Possible Fix: I could run a separate function to count all the keys and use a counter as my control in a simple for loop, rather than letting forEach handle the control. That seems a bit inefficient, no?
Here's the code:
function fixJsonKeys(obj, results) {
async.forEach(Object.keys(obj), function(key, callback) {
if (typeof obj[key] == 'object') {
// do stuff to json key, then call the function
// on the nested object
fixJsonKeys(obj[key]);
callback(); // <-- how does this work with recursion??
}
else {
// do stuff to json key
callback();
}
}, function(err) {
if (err) return next(err);
// obj keys fixed, now return completed object
results(obj);
});
}
EDIT: hard to format in comments, so:
#Laksh and #Suhail: I tried your suggestions and same outcomes. If I remove the callback from the if condition, the async.forEach still looks to confirm that it has handled the top level keys (I think).
For example, say I have 3 top level keys, and one of them has a nested object:
[key1] (no callback, do recursion)
--[subKey1]
--[subKey2]
[key2] (callback)
[key3] (callback)
async.forEach is still looking for a callback on action taken for key1. Do I have this correct?
Since you are using recursive functions here, you don't need to call callback() inside the if condition.
This is because your recursive call will return to the existing callback() in if once the entire recursive stack finished for that particular obj[key].
Recursive function return only when the base condition is true, in your case when if condition fails so it will automatically call callback() from else block.
I have to change the value of an json element, if it exists, before making an async function call, but I'm not sure if this code will execute in the order that I want. Here is a simple example:
function first_function(json, callback) {
get_only_valid_elements_json(json, function(valid_json) {
if (valid_json.element) {
valid_json.element = something;
}
update_something(valid_json, function(error) {
if (error) {
return callback(error);
} else {
return callback({success:"YAY"});
}
});
});
}
I would like to know how the following code will execute and more important why. Thanks!
It looks right to me.
Assuming get_only_valid_elements_json, and update_something call the function passed to them when they're finished with their job, this is how it is executed:
it calls get_only_valid_elements_json, and first_function function returns
when get_only_valid_elements_json finishes (i.e. when it calls the callback function passed in the second argument), it will execute the if statement, and conditionally, its block
then, update_something is called and then function(valid_json) {...} returns
when update_something finishes (i.e. calls the callback function), it will execute the if (error) statement and the appropriate block
the callback function is called (either with error or success), signalling that get_only_valid_elements_json finished its job
So the functions synchronously return before their "real job" is done.
The reason for asynchronous calls is to not block the executing process when you're waiting for something, e.g. a response to an AJAX call. You just tell the browser to execute a given function when the response is received, so when it receives the response it calls the function you gave to the AJAX call, which calls the function given to update_something, which then calls the callback given to first_function.
I was trying to solve the "Juggling Async" problem in the "learnyounode" workshop from nodeschool.io. I saw a lot of questions here about this problem where it is not working because the url's were being called from a loop. I understand why that wouldn't work.
I had tried something like this.
var http=require('http');
console.log(process.argv[2],process.argv[3],process.argv[4]);
fn(process.argv[2],
fn(process.argv[3],
fn(process.argv[4])));
function fn(url,callback){
http.get(url,function(response){
var string='';
response.setEncoding('utf8');
response.on('data',function(data){
string+=data;
});
response.on('end',function(){
console.log(url,string);
if (callback) {
callback();
}
});
});
};
As per my understanding, the second GET call should go out only after the first one has ended, since it is being initiated after response end. But the output is always in different order.
I have seen the correct solution. I know that doing it this way fails to leverage the advantage of async, but shouldn't the callbacks make it output in order?
When you pass in a function as a callback, you need to only pass in the function name or function definition. By providing the arguments, you are actually calling the function.
E.g. let's say you had a function f1 that took in another function as a parameter, and another function f2 that you want to pass into f1:
function f1(func_param) {
console.log('Executing f1!');
}
function f2(a_param) {
console.log('Executing f2!');
}
When you do the following call (which is similar to what you are doing, providing a callback function and specifying parameters for the callback):
f1(f2(process.argv[2]));
You're evaluating f2 first, so 'Executing f2!' will print first. The return of f2 will get passed as a parameter into f1, which will execute and print 'Executing f1!'
In your call
fn(process.argv[2],
fn(process.argv[3],
fn(process.argv[4])));
You are saying, let's pass the result of fn(process.argv[4]) as a callback to fn(process.argv[3]), and we'll get the result of calling that and pass that as a callback to fn(process.argv[2]).
I have the node.js code running on a server and would like to know if it is blocking or not. It is kind of similar to this:
function addUserIfNoneExists(name, callback) {
userAccounts.findOne({name:name}, function(err, obj) {
if (obj) {
callback('user exists');
} else {
// Add the user 'name' to DB and run the callback when done.
// This is non-blocking to here.
user = addUser(name, callback)
// Do something heavy, doesn't matter when this completes.
// Is this part blocking?
doSomeHeavyWork(user);
}
});
};
Once addUser completes the doSomeHeavyWork function is run and eventually places something back into the database. It does not matter how long this function takes, but it should not block other events on the server.
With that, is it possible to test if node.js code ends up blocking or not?
Generally, if it reaches out to another service, like a database or a webservice, then it is non-blocking and you'll need to have some sort of callback. However, any function will block until something (even if nothing) is returned...
If the doSomeHeavyWork function is non-blocking, then it's likely that whatever library you're using will allow for some sort of callback. So you could write the function to accept a callback like so:
var doSomHeavyWork = function(user, callback) {
callTheNonBlockingStuff(function(error, whatever) { // Whatever that is it likely takes a callback which returns an error (in case something bad happened) and possible a "whatever" which is what you're looking to get or something.
if (error) {
console.log('There was an error!!!!');
console.log(error);
callback(error, null); //Call callback with error
}
callback(null, whatever); //Call callback with object you're hoping to get back.
});
return; //This line will most likely run before the callback gets called which makes it a non-blocking (asynchronous) function. Which is why you need the callback.
};
You should avoid in any part of your Node.js code synchronous blocks which don't call system or I/O operations and which computation takes long time (in computer meaning), e.g iterating over big arrays. Instead move this type of code to the separate worker or divide it to smaller synchronous pieces using process.nextTick(). You can find explanation for process.nextTick() here but read all comments too.