AWSERED DONT NEED TO WRITE
How to make this loop run synchronous in node js
CODE:
for(var i = 0; i < arr.length; i++) {
inv(arr[i][0],arr[i][1]);
}
inv(arr[i][0],arri); // this is a function which fetches data from a website must be executed one by one or ip will be blocked cant be executed async
What program does:image
the problem was with the function it was async so i made it recursive to solve the problem
Related
I am writing a nodejs script to process (convert to PDF from various formats) large amount of files on regular basis. The script reads all files in the folder, splits them into even sized arrays and then is supposed to run a command line converter asynchronously on each of the arrays. Everything works fine except for the node waiting for the promise to return before continuing with the for loop. The code uses a test command for copying files instead of conversion because it's simpler for testing. Instead of running 5 simultaneous copy tasks it runs one after the other. I am using array of promises so I can call Promises.all()when all promises return. What am I missing that this is not working as it should?
let processCount = 5;
let promisesArray = [];
for (let processNo = 0; processNo < processCount; processNo++) {
promisesArray[processNo] = new Promise(function(resolve, reject) {
let fileList = splitArrays[processNo];
for (let file = 0; file < fileList.length; file++) {
let command = `Copy "${fileList[file][0]}" "${fileList[file][1]}"`;
execSync(command);
}
resolve(true);
});
}
execSync() by definition blocks the event loop until the child process is done. You'll have to use exec() instead. The third param of exec() is callback function which will be called when your child process exits.
Recently I came across code that makes me wonder about how Node.js engine works. Specifically in regards to event loop, for loop, and asynchronous processing. For example:
const x = 100000000; // or some stupendously large number
for (let i = 0; i < x; i++) {
asyncCall(callback);
}
const callback = () => {
console.log("done one");
}
Let's say asyncCall takes anywhere from 1ms to 100ms. Is there a possibility that console.log("done one") will be called before the for loop finishes?
To try to find my own answer, I've read this article: https://blog.sessionstack.com/how-javascript-works-event-loop-and-the-rise-of-async-programming-5-ways-to-better-coding-with-2f077c4438b5. But I'm not sure if there is a case where the call stack will be empty in the middle of the for loop so that the event loop puts the callback in between asyncCall calls?
How can I run a single method multiple times multi-threaded when called as a method of a class?
At first I tried to use the cluster module, but I realize it just re-runs the whole process from the start, rightfully so.
How can I achieve something like what's outlined below?
I want a class's method to spawn n processes, and when the parallel tasks are completed, I can resolve a promise which the method returns.
The problem with the code below is that calling cluster.fork() will fork index.js process.
index.js
const Person = require('./Person.js');
var Mary = new Person('Mary');
Mary.run(5).then(() => {...});
console.log('I should only run once, but I am called 5 times too many');
Person.js
const cluster = require('cluster');
class Person{
run(distance){
var completed = 0;
return new Promise((resolve, reject) => {
for(var i = 0; i < distance; i++) {
// run a separate process for each
cluster.fork().send(i).on('message', message => {
if (message === 'completed') { ++completed; }
if (completed === distance) { resolve(); }
});
}
});
}
}
I think the short answer is impossible. It's even worse - this has nothing to do with js. To multi (process or thread) in your particular problem you will essentially need a copy of the object in every thread, since it needs (maybe) access to fields - in this case you would need to either initialize it in every thread or share memory. That last one I don't think is provided in cluster, and not trivial in other languages in every use case.
If the calculation is independent of the Person I suggest you extract it, and use the usual (in index.js):
if(cluster.isWorker) {
//Use the i for calculation
} else {
//Create Person, then fork children in for loop
}
You then collect the results and change the Person as needed. You will be copying index.js, but this is standard and you only run what you need.
The problem is if results are dependent on Person. If these are constant for all i you can still send them to your forks independently. Otherwise what you have is the only way to fork. In general forking in cluster is not meant for methods, but for the app itself, which is the standard forking behavior.
Another solution
Following your comment, I suggest you checkout child_process.execFile or child_process.exec on same file.
This way you can spawn a totally independent process on the fly. Now instead of calling cluster.fork you call execFile. You can use either the exit code or stdout as return values (stderr etc.). Promise is now replaced with:
var results = []
for(var i = 0; i < distance; i++) {
// run a separate process for each
results.push(child_process.execFile().child.execFile('node', 'mymethod.js`,i]));
}
//... catch the exit event from all results or return a callback using results.
Inside mymethod.js Have your code that takes i and returns what you want either in the exit code or through stdout, both properties of the returned child_process. This is a bit un-node.js-y since you're waiting on asynchronous calls, but you're requirements are non standard. Since I'm not sure how you use this perhaps returning a callback with the array is a better idea.
This question already has answers here:
How node.js server serve next request, if current request have huge computation?
(2 answers)
NodeJs how to create a non-blocking computation
(5 answers)
Closed 5 years ago.
All:
I am pretty new to Node async programming, I wonder how can I write some Express request handler which can handle time consuming heavy calculation task without block Express handling following request?
I thought setTimeout can do that to put the job in a event loop, but it still block other requests:
var express = require('express');
var router = express.Router();
function heavy(callback){
setTimeout(callback, 1);
}
router.get('/', function(req, res, next) {
var callback = function(req, res){
var loop = +req.query.loop;
for(var i=0; i<loop; i++){
for(var j=0; j<loop; j++){}
}
res.send("finished task: "+Date.now());
}.bind(null, req, res);
heavy(callback)
});
I guess I did not understand how setTimeout works(my understanding about setTimeout is after that 1ms delay it will fire up the callback in a seperated thread/process without blocking other call of heavy), could any one show me how to do this without blocking other request to heavy()?
Thanks
Instead of setTimeout it's better to use process.nextTick or setImmediate (depending od when you want your callback to be run). But it is not enough to put a long running code into a function because it will still block your thread, just a millisecond later.
You need to break your code and run setImmediate or process.nextTick multiple times - like in every iteration and then schedule a new iteration from that. Otherwise you will not gain anything.
Example
Instead of a code like this:
var a = 0, b = 10000000;
function numbers() {
while (a < b) {
console.log("Number " + a++);
}
}
numbers();
you can use code like this:
var a = 0, b = 10000000;
function numbers() {
var i = 0;
while (a < b && i++ < 100) {
console.log("Number " + a++);
}
if (a < b) setImmediate(numbers);
}
numbers();
The first one will block your thread (and likely overflow your call stack) and the second one will not block (or, more precisely, it will block your thread 10000000 times for a very brief moment, letting other stuff to run in between those moments).
You can also consider spawning an external process or writing a native add on in C/C++ where you can use threads.
For more info see:
How node.js server serve next request, if current request have huge computation?
Maximum call stack size exceeded in nodejs
Node; Q Promise delay
How to avoid jimp blocking the code node.js
NodeJS, Promises and performance
I am trying to run a function in a async way, but I always recive the result in the order that i call them instead of receive the quickest function.
This is an example. The expected result is to print first 100 even it is called last.
function f1(callback){
i = 0;
for (i;i< 100;i++){
}
if (i == 100){
return callback(i);
}
}
function f2(callback){
i = 0;
for (i;i< 99999999999;i++){
}
if (i == 99999999999){
return callback(i);
}
}
f2(function(i){
console.log(i);
})
f1(function(i){
console.log(i);
});
In node (and in javascript in general), functions are not asynchronous by default.
In your code above, at no point are you yielding control to the javascript's event loop. As you call each function, it runs through its for loop until finished and then calls back which is why you are seeing the output the way you are.
The asynchronous nature of node kicks in when you start doing IO or purposely yield control to the VM and give it a chance to process other events off the event loop.
To see the code above run in an asynchronous fashion for the purposes of your test:
function f1(callback){
setTimeout(callback('f1'), 100);
}
function f2(callback){
setTimeout(callback('f2'), 500);
}
f2(function(i){
console.log(i);
})
f1(function(i){
console.log(i);
});
The setTimeout call basically just says: call back to the provided function as close to n milliseconds as you can (but not before). Basically acting as a sleep statement here and yielding control back to the currently executing code.