Much appreciated if someone can give a more concrete example of how to use the async.applyEachSeries function.
async.applyEach([enableSearch, updateSchema], 'bucket', callback);
// partial application example:
async.each(
buckets,
async.applyEach([enableSearch, updateSchema]),
callback
);
This is from the async readme documentation, but I have no idea how to use it. What is the 'bucket', is it just a string being passed to somewhere?
async.applyEach([enableSearch, updateSchema], 'bucket', callback);
This calls enableSearch('bucket') and updateSearch('bucket') asynchronously with callbacks, and then calls callback() once they are both done.
// partial application example:
async.each(
buckets,
async.applyEach([enableSearch, updateSchema]),
callback
);
The async.applyEach() call returns a function that calls both enableSearch() and updateSchema() with its arguments when it is called. Since the first argument to async.each() should be an Array, I assume that buckets is an Array of Strings (as the first example involved passing one string to enableSearch() and updateSearch()). As a result, this could would call enableSearch() and updateSearch() with each String in the Array buckets (and then call callback()).
Here's a pretty clear example of async.applyEachSeries:
applyEachSeries(tasks, args..., [callback])
example:
function one(arg1, arg2, callback) {
// Do something
return callback();
}
function two(arg1, arg2, callback) {
// Do something
return callback();
}
async.applyEachSeries([one, two], 'argument 1', 'argument 2', function finalCallback(err) {
// This will run after one and two
});
Source: https://stackoverflow.com/a/35309495
Related
I have a library that makes a web request and passes the results to a callback like the sample below:
someReadyWebSearchThirdPartyLibrary.getSearch(parameters, error, callback);
Then the callback is something like:
var callback = function (data) {
console.log(data);
};
Is there a way I can pass more arguments to the callback from the caller given this is an off the shelf ready library?
just define your "first" callback inline, there you can pass more parameters:
someReadyWebSearchThirdPartyLibrary.getSearch(parameters, error, function(data) {
callback(data, otherVar1, otherVar2);
});
Then your callback could look something like this:
var callback = function (data, otherVar1, otherVar2) {
console.log(data);
console.log(otherVar1);
console.log(otherVar2);
};
Without editing the code of the library that calls the callback, you can't change what is passed to that specific callback. But, there are multiple options where you can get access to other variables:
Inline callback accessing variables in scope
you can use scope and an inline definition to make other variables available (via scoping) to the callback.
let someVar1 = "foo";
let someVar2 = 4;
someReadyWebSearchThirdPartyLibrary.getSearch(parameters, error, function(data) {
// can access data, someVar1 and someVar2 here as they are all in scope
console.log(someVar1);
});
Use .bind() to add arguments to the callback
You could also use .bind() to create a new function that adds parameters.
function callback(someVar1, someVar2, data) {
console.log(someVar1);
}
someReadyWebSearchThirdPartyLibrary.getSearch(parameters, error, callback.bind(null, "foo", 4));
.bind() creates a new function that adds additional arguments before calling the actual callback() function.
Make your own stub function
Or, make your own stub function that calls your callback (essentially what the .bind() option was doing for you):
function mycallback(someVar1, someVar2, data) {
console.log(someVar1);
}
someReadyWebSearchThirdPartyLibrary.getSearch(parameters, error, function(data) {
mycallback("foo", 4, data);
});
I assume your callback arguments are variable and that the function is defined in an isolated manner (where it can't access scope beyond itself, and that you don't want to mutate the callback function's underlying data. It also assumes that Data is an object (You should perform some validation on this though). Therefore you could do something like the following:
someReadyWebSearchThirdPartyLibrary.getSearch(parameters, error, function(data) {
data.newArgument = 'lorem';
callback(data);
});
This way you can add any arbitrary arguments onto the data object passed.
I'm trying to build a bot cluster, and I'm having some trouble here.
I'd like to call the following function, called BuildABot.
http://pastebin.com/zwP7rZay
Now after this call is finished, I'd like to call requestItems, which takes in the steamOffers object as a parameter.
http://pastebin.com/jsJ4fhwG
However, the object is null, because the call hasn't finished.
Is there any way to halt the call until buildABot is done?
There are various ways to handle your requirement, I will mention 2 options while I am sure you can find more. You may find many more examples all over the web.
Option 1: Using a callback function -
Pass a callback function to the async function, When async function is finished it will call the callback func. This way the code in the callback func will be executed only when async call is over.
Related topic - how-to-write-asynchronous-functions-for-node-js
Option 2: If you have more complicated logic and you want to perform one part after another you can use the async waterfall.
Code example:
async.waterfall([
function(callback){
callback(null, 'one', 'two');
},
function(arg1, arg2, callback){
// arg1 now equals 'one' and arg2 now equals 'two'
callback(null, 'three');
},
function(arg1, callback){
// arg1 now equals 'three'
callback(null, 'done');
}
], function (err, result) {
// result now equals 'done'
});
See async module site for more information.
I need to call 3 functions in series x1() then x2() then x3(). But since x1 and x2 are time consuming operations x3 executes before them giving unexpected results. I have used the async library in NodeJS with the series method as below.
How can I solve this without using setTimeout for x3()
async.series([function(callback) { x1();callback(null, null);},
function(callback) { x2();callback(null, null);},
function(callback) {
x3();
callback(null, null); }],
function(err, results) { }
);
use recursive try catch..
a=true;
while(a){
try{
your work.....
a=false;
}catch(exp ){
a=true
}
}
You can use javascript's async nature to solve it. Here how it's done with callbacks. Call x3() instead of onMethodsDone().
Here, suppose x1() accepts an array. Forget it if you don't need any parameters. x3() gets the modified array from x2() which is also another time consuming function. setTimeout() is used to show the async behaviour since x1() and x2() are time consuming.
x1([1,2],function(results){
x2(results,onMethodsDone);
});
function onMethodsDone(results){
alert(results);
}
function x1(records,cb){
setTimeout(function (){
records.push(3,4,5);
cb(records); //parse updated records to callback
},2000);
}
function x2(records,cb){
setTimeout(function (){
records.push(6,7,8);
cb(records);
},2000);
}
javascripts Listeners or Promise/defers also can be used to handle it.
As the callback function had to be overloaded with different number of arguments the best option was to structure the method as follows
function callback(cbargs){
var arg1 = cbargs.cbarg1;
var arg2 = cbargs.cbarg2;
if(arg3!=null){
.....
}
}
function x1(arg1 , arg2, callback, cbargs){
.....
.....
callback(cbargs);
}
function x3(arg1 , arg2, callback, cbargs){
.....
.....
callback(cbargs);
}
x1(arg1, arg2,callback,{cbarg1, cbarg2});
x1(arg1, arg2,callback,{cbarg1, cbarg2, cbarg3});
I have two problems implementing a RESTful service using Node.js / node-postgres lib / PostgreDB and both are due to the async nature of JS.
A) I need to pass an extra argument to a callback in client.query(query, callback) call
I am inside a callback of a query and going through an array of recently fetched rows from a DB and want to launch a subsequent query for each of them:
var query = client.query('SELECT * FROM event', queryAllEventsHandler);
function queryAllEventsHandler(err, result){
allEvents = result.rows;
/* allEvents is an JSON array with the following format
[ {"id_event":1, "name":"name of the event"},
{"id_event":1, "name":"name of the event"}
]
*/
for(var i = 0; i<allEvents.length; i++){
client.query('SELECT * FROM days where id_event = $1',[allEvents[i].id_event], function( err, result){
//I want to have a reference to variable i
}
}
In the above example I want to do something like:
client.query('SELECT * FROM days where id_event = $1',[allEvents[i].id_event], function( AN_EXTRA_ARG, err, result)
Where the AN_EXTRA_ARG is an extra argument or a closure in the callback function... How can I achieve this? Should I create an closure with the of i and pass it as a callback's arg? How ? :|
B) "Synchronizing" queries
I need to launch various queries and create a custom JSON from all of them. Since every query and it's callback are asynchronous (waiting for no one) I was looking for a way to "tame" it and among other stuff I found a solution that occured to me in the first place, but seemed a bit "bad/lousy":
Keeping the query count is really the way to go as #jslatts suggests in Synchronous database queries with Node.js?
Hope I
With regards to question A, you could create a function to handle both your queries and only return when the last query is executed and return both results to the callback.
for(var i = 0; i<allEvents.length; i++){
query(client, allEvents[i], function(result1, result2) {
//do something
});
}
function query(client, event, callback) {
client.query('SELECT * FROM days where id_event = $1',[event.id_event], function( err1, result1){
client.query('SELECT * FROM days where id_event = $1',[event.id_event], function( err2, result2){
callback(result1, result2);
});
});
}
I don't like answering my on question, but this might be of interest to someone.... Regarding the A part of my question. You can assign a custom object to this in your function.
As you know a keyword this corresponds to the Window (top) object when inside a function (unless it's a method function). Using the bind function you can change the reference of this to your own object...
So what I did was, I created a named function queryCallback
function queryCallback(err, result){
//this == Window (default)
}
changed the anonymous callback function to the named one queryCallback:
client.query('SELECT * ... where id_event = $1',[allEvents[i].id_event], queryCallback.bind( {"position":i}, err, result));
Now, note queryCallback.bind( {"position":i}, err, result));
What bind(my_custom_this, [other args]) does is it binds a custom object (in my case {"position":i}) to this inside the function upon which the bind was called...
Now we have this scenario:
function queryCallback(err, result){
//this == {"position":i}
}
Bind explained: http://fitzgeraldnick.com/weblog/26/
A) I personally like lodash (or underscore if you prefer) partial() for this. It takes a function and a number of arguments and returns a function with the provided arguments applied and the remaining arguments still open. It's very much like the functional concept of currying.
B) For combining multiple asynchronous results I highly recommend async. The syntax will take a little getting used to, but makes thing like this very easy. Quick sample:
async.parallel([
one: function(callback){
db.fetch(options, callback);
},
two: function(callback){
db.fetch(options, callback);
}
],
function(err, results){
// this callback will get called when either parallel call gives an error
// or when both have called the callback
if (err) {
// handle error
return;
}
// get the results from results.one and results.two
});
== Added in edit ==
Actually lodash also provides a nicer (imho) albeit slightly more expensive (due to function calls) solution for your problem A):
_(allEvents).each(function(event, index, array) {
client.query('SELECT * FROM days where id_event = $1',[event.id_event], function( err, result) {
// Just use 'index' here, which doesn't change during the each
}
});
For B), your options include async or via a Promise library (such as Q, when.js, Bluebird, etc...)
I'm trying to use async and request module together but i don't understand how the callbacks get passed. My code is
var fetch = function(file, cb) {
return request(file, cb);
};
async.map(['file1', 'file2', 'file3'], fetch, function(err, resp, body) {
// is this function passed as an argument to _fetch_
// or is it excecuted as a callback at the end of all the request?
// if so how do i pass a callback to the _fetch_ function
if(!err) console.log(body);
});
I'm trying to fetch 3 files in order and concatenate the results. My head is stuck in callbacks I tryed and the different combinations I could think of. Google wasn't much help.
Request is asynchronous function, it does not return something, when its job is done, it calls back. From request examples, you should do something like:
var fetch = function(file,cb){
request.get(file, function(err,response,body){
if ( err){
cb(err);
} else {
cb(null, body); // First param indicates error, null=> no error
}
});
}
async.map(["file1", "file2", "file3"], fetch, function(err, results){
if ( err){
// either file1, file2 or file3 has raised an error, so you should not use results and handle the error
} else {
// results[0] -> "file1" body
// results[1] -> "file2" body
// results[2] -> "file3" body
}
});
In your example, the fetch function will be called three times, once for each of the file names in the array passed as the first parameter to async.map. A second, callback parameter will also be passed into fetch, but that callback is provided by the async framework and you must call it when your fetch function has completed its work, providing its results to that callback as the second parameter. The callback you provide as the third parameter to async.map will be called when all three of the fetch calls have called the callback provided to them.
See https://github.com/caolan/async#map
So to answer your specific question in the code, the callback function you provide is executed as a callback at then end of all requests. If you need to pass a callback to fetch you'd do something like this:
async.map([['file1', 'file2', 'file3'], function(value, callback) {
fetch(value, <your result processing callback goes here>);
}, ...