So in Node.js let's say I have the following code:
for (var i = 0; i < 1000; i++) {
someAsynchronousFunction(function(err,res) {
// Do a bunch of stuff
callback(null,res);
});
}
But I want this to run synchronously. I know this is not advised in Node JS, but I am just trying to understand the language. I tried implementing the following solution, but it just ends up hanging during runtime:
for (var i = 0; i < 1000; i++) {
var going = true;
someAsynchronousFunction(function(err,res) {
// Do a bunch of stuff
callback(null,res);
going = false;
});
while (going) {
}
}
What is going wrong and what is the correct way to do this?
One of the best way to do that is to use the async library.
async.timesSeries(1000, function(n, next){
someAsynchronousFunction(next);
});
Or you can do that with async.series() function.
.times() documentation : http://caolan.github.io/async/docs.html#.timesSeries
Another way to do this is using Promises to generate a sequential execution of them thanks to Array#reduce :
// Function that returns Promise that is fllfiled after a second.
function asyncFunc (x){
return new Promise((rs, rj)=>{
setTimeout( ()=>{
console.log('Hello ', x);
rs();
}, 1000)
});
}
// Generate an array filed with values : [0, 1, 2, 3, ...]
Array.from({length : 1000}, (el, i)=> i)
// loop througth the array chaining the promises.
.reduce( (promise, value) =>
promise.then(asyncFunc.bind(null, value))
, Promise.resolve(null));
Related
Please find below code
function get_btc(address) {
address_transaction(address, user_id, coin_key, deposite_txn_fee, function(callback) {
for (var j = 0; j < callback.response.data.txs.length; j++) {
let user_id = callback.user_id;
//some code//
}
});
}
get_label_info(function(err, data) {
for (var i = 0; i < data.length; i++) {
let address = data[i].address;
deposite_model.get_coin_info(function(err, data1) {
var coin_name = data1[0].coin_code;
const return_functions = get_switch(coin_name);
if (return_functions) {
obj[return_functions](address);
}
})
}
});
function all_completed() {
console.log('all functions has been completed');
}
By the help of above mentioned code i want to excecute all_completed loop when all functions has been completly done.
At the initial start get_label_info function is excuted then controller go on to get_btc function.
Please help me how could i run all_completed functions after all functions completed run.
I'll assume you are using es6, and that you know what a Promise is in that context. In that case wrap all your callback based things in a Promise that resolves when the callback completes. Then, in your loop, push all your Promises into an array variable. Finally call Promise.all with that array as an argument and call then on the result to encapsulate the code you want ti run after they all complete (resolve).
I'm trying to loop & sort a large number of data ( the whole ethereum blockchain lol )
I'm trying to create a record of all transactions for every address.
Obviously this is a very intensive process and I'm not sure how to make it more efficient beyond what I have (which isn't that efficient)
It starts out quick but I'm thinking now it has slowed because of the lookup for the address in the txs object.
Any help opinions / help is greatly appreciated.
https://giphy.com/gifs/3o6fJ7KWqxESY9okk8
var txs = {};
var i = 0;
// Loop over blocks
(function loop () {
setTimeout(function () {
// Get current block
var block = web3.eth.getBlock(i, true, (error, block) => {
// debugger;
// Loop over transactions in block
for(var j = 0; j < block.transactions.length; j++) {
// debugger;
if(txs[block.transactions[j].to]) {
txs[block.transactions[j].to].transactions.push(block.transactions[j]);
} else if (txs[block.transactions[j].to]) {
txs[block.transactions[j].from].transactions.push(block.transactions[j]);
} else {
txs[block.transactions[j].to] = {
transactions: [block.transactions[j]]
}
txs[block.transactions[j].from] = {
transactions: [block.transactions[j]]
}
}
}
});
i++
if (i < highestBlock) {
loop();
}
}, 50);
})();
I think that your code has an error in it located at "else-if", it seems that you should use the txs[block.transactions[j].from] property instead of the txs[block.transactions[j].to]. If you simply want to accomplish a recursive pattern you could use the setImmediate function or the process.nextTick method. If you use node.js v6+ you could use a Map instead of the object.
Is there a way to make Node.js stream as coroutine.
Example
a Fibonacci numbers stream.
fibonacci.on('data', cb);
//The callback (cb) is like
function cb(data)
{
//something done with data here ...
}
Expectation
function* fibonacciGenerator()
{
fibonacci.on('data', cb);
//Don't know what has to be done further...
};
var fibGen = fibonacciGenerator();
fibGen.next().value(cb);
fibGen.next().value(cb);
fibGen.next().value(cb);
.
.
.
Take desired numbers from the generator. Here Fibonacci number series is just an example, in reality the stream could be of anything a file, mongodb query result, etc.
Maybe something like this
Make the 'stream.on' function as a generator.
Place yield inside the callback function.
Obtain generator object.
Call next and take the next value in stream.
Is it at-least possible if yes how and if not why? Maybe a dumb question :)
If you don't want to use a transpiler (e.g. Babel) or wait until async/await make it to Node.js, you can implement it yourself using generators and promises.
The downside is that your code must live inside a generator.
First, you can make a helper that receives a stream and returns a function that, when called, returns a promise for the next "event" of the stream (e.g. data).
function streamToPromises(stream) {
return function() {
if (stream.isPaused()) {
stream.resume();
}
return new Promise(function(resolve) {
stream.once('data', function() {
resolve.apply(stream, arguments);
stream.pause();
});
});
}
}
It pauses the stream when you're not using it, and resumes it when you ask it the next value.
Next, you have a helper that receives a generator as an argument, and every time it yields a promise, it resolves it and passes its result back to the generator.
function run(fn) {
var gen = fn();
var promise = gen.next().value;
var tick = function() {
promise.then(function() {
promise = gen.next.apply(gen, arguments).value;
}).catch(function(err) {
// TODO: Handle error.
}).then(function() {
tick();
});
}
tick();
}
Finally, you would do your own logic inside a generator, and run it with the run helper, like this:
run(function*() {
var nextFib = streamToPromises(fibonacci);
var n;
n = yield nextFib();
console.log(n);
n = yield nextFib();
console.log(n);
});
Your own generator will yield promises, pausing its execution and passing the control to the run function.
The run function will resolve the promise and pass its value back to your own generator.
That's the gist of it. You'd need to modify streamToPromises to check for other events as well (e.g. end or error).
class FibonacciGeneratorReader extends Readable {
_isDone = false;
_fibCount = null;
_gen = function *() {
let prev = 0, curr = 1, count = 1;
while (this._fibCount === -1 || count++ < this._fibCount) {
yield curr;
[prev, curr] = [curr, prev + curr];
}
return curr;
}.bind(this)();
constructor(fibCount) {
super({
objectMode: true,
read: size => {
if (this._isDone) {
this.push(null);
} else {
let fib = this._gen.next();
this._isDone = fib.done;
this.push(fib.value.toString() + '\n');
}
}
});
this._fibCount = fibCount || -1;
}
}
new FibonacciGeneratorReader(10).pipe(process.stdout);
Output should be:
1
1
2
3
5
8
13
21
34
55
I want to fill each object of the result of a query, with other querys, and I want to do all in asynchronously way
Here is an example of the way how I do actually
var q = knex.select().from('sector');
q.then(function (sectores) {
var i = -1;
(function getDetalles(sectores) {
i++;
if(i < sectores.length){
knex.select().from('sector_detalle')
.where('sector_id', sectores[i].id)
.then(function (detalles) {
// this what i want to do asynchronously
sectores[i].sector_detalles = detalles;
console.log(sectores[i]);
getDetalles(sectores);
});
} else {
res.send({sucess: true, rows: sectores});
}
})(sectores);
});
I do some reserch and found this wait for all promises to finish in nodejs with bluebird
is close to what I want but don't know how to implement
I think you're looking for the map method that works on a promise for an array, and will invoke an asynchronous (promise-returning) callback for each of the items in it:
knex.select().from('sector').map(function(sector) {
return knex.select().from('sector_detalle')
.where('sector_id', sector.id)
.then(function(detalles) {
sector.sector_detalles = detalles;
// console.log(sector);
return sector;
});
}).then(function(sectores) {
res.send({sucess: true, rows: sectores});
});
Is there any way to get generators into node.js?
I'm currently faking them with callbacks, but I have to remember to check the response of the callback inside of my generator function which creates a lot of if (callback(arg) === false) return;
I want something like in python:
for p in primes():
if p > 100: break
do_something(p)
which I'm doing in node like this:
primes(function(p) {
if (p > 100) return false;
do_something(p)
});
Maybe something like coffeescript could help?
Yes, since version 0.11. Enjoy!
http://wingolog.org/archives/2013/05/08/generators-in-v8
http://jlongster.com/A-Study-on-Solving-Callbacks-with-JavaScript-Generators
The answer is "not currently" but Marcel seems to be my hero. Lets hope this goes somewhere:
https://groups.google.com/forum/#!msg/nodejs/BNs3OsDYsYw/oCsWBw9AWC0J
https://github.com/laverdet/node-fibers
You can use generators in Node.js, but only in 0.11+. Node.js 0.12 (stable) is now available. Add --harmony_generators or --harmony to the command line parameters of node to enable it.
With Traceur, you can compile advanced JavaScript to vanilla JavaScript. You could make a loader for node.js that does this on-the-fly. Since it runs on, and compiles to vanilla JavaScript, it runs in node.js < 0.11 as well as in the browser.
Facebook has developed a lighter version that only supports generators, called Regenerator. It works similarly to Traceur.
Apparently not in the current stable version. You can however achieve the same using node-fibers + promises.
Here is my implementation:
var fiber = require('fibers');
module.exports.yield = function (promise) {
var currentFiber = fiber.current;
promise
.then(function (value) {
currentFiber.run(value);
})
.otherwise(function (reason) {
currentFiber.throwInto(reason);
});
return fiber.yield();
};
module.exports.spawn = function (makeGenerator) {
fiber(function () {
makeGenerator.apply(this, Array.prototype.slice.call(arguments, 1));
}).run();
};
And a sample code on how it works: (query.find returns a promise)
var generators = require('./utils/generators');
var query = require('./utils/query');
generators.spawn(function () {
try {
var field1 = generators.yield(query.find('user', { _id : '1' }));
var field2 = generators.yield(query.find('user', { _id : '2' }));
console.log('success', field1[0]._id, field2[0]._id);
}
catch (e) {
console.error('error', e);
}
});
You might check out wu.js at http://fitzgen.github.com/wu.js/ It has lots of interesting iterator functions.
Yes and no.
var myGen = (function () {
var i = 0;
return function () {
i++; return i; }
})();
var i;
while ((i = myGen()) < 100 ) {
do something; }
As you see, you can implement something like one using closures, but it does not have native generators.
The issue proposing generatiors in v8 has recently been accepted by v8 project member.
Please vote there to make yield come true.
Update 2014: Node does support callbacks now. The following is a post from 2010.
You should use callbacks. If the function does something asynchronously, you may also want a continuation callback (continuation is a bad word, since it also means something else, but you get my point.)
primes(function(p) {
if (p > 100) return false // i assume this stops the yielding
do_something(p)
return true // it's also better to be consistent
}, function(err) { // fire when the yield callback returns false
if (err) throw err // error from whatever asynch thing you did
// continue...
})
Updated with example code
I flipped it, so that it returns true on complete (since null, false and undefined all evaluate to false anyways).
function primes(callback) {
var n = 1, a = true;
search: while (a) {
n += 1;
for (var i = 2; i <= Math.sqrt(n); i += 1)
if (n % i == 0)
continue search;
if (callback(n)) return
}
}
primes(function(p) {
console.log(p)
if (p > 100) return true
})
We are using gnode for generators in node < 0.11.3 - https://github.com/TooTallNate/gnode
Yes Node.js and JavaScript now have both synchronous iterators (as of atleast Node v6) and asynchronous iterators (as of Node v10):
An example generator/iterator with synchronous output:
// semi-pythonic like range
function* range(begin=0, end, step=1) {
if(typeof end === "undefined") {
end = begin;
begin = 0;
}
for(let i = begin; i < end; i += step) {
yield i;
}
}
for(const number of range(1,30)) {
console.log(number);
}
A similar async generator/iterator.
const timeout = (ms=1000) => new Promise((resolve, reject) => setTimeout(resolve, ms));
async function* countSeconds(begin=0, end, step=1) {
if(typeof end === "undefined") {
end = begin;
begin = 0;
}
for(let i = begin; i < end; i += step) {
yield i;
await timeout(1000);
}
}
(async () => {
for await (const second of countSeconds(10)) {
console.log(second);
}
})();
There is a lot to explore here are some good links. I will probably update this answer with more information later:
Generators
Generator functions
Iterable Protocol
Iterator Protocol
Async Generators
Jake Archibald's Article on Async Generators
Async Iterators
for await ... of