I have this code periodically calls the load function which does very load work taking 10sec. Problem is when load function is being executed, it's blocking the main flow. If I send a simple GET request (like a health check) while load is being executed, the GET call is blocked until the load call is finished.
function setLoadInterval() {
var self = this;
this.interval = setInterval(function doHeavyWork() {
// this takes 10 sec
self.load();
self.emit('reloaded');
}, 20000);
I tried async.parallel but still the GET call was blocked. I tried setTimeout but got the same result. How do I make load to running on background so that it doesn't block the main flow?
this.interval = setInterval(function doHeavyWork() {
async.parallel([function(cb) {
self.load();
cb(null);
}], function(err) {
if (err) {
// log error
}
self.emit('reloaded');
})
}, 20000);
Node.js is an event driven non-blocking IO model
Anything that is IO is offloaded as a separate thread in the underlying engine and hence parallelism is achieved.
If the task is CPU intensive there is no way you can achieve parallelism as by default Javascript is a blocking sync language
However there are some ways to achieve this by offloading the CPU intensive task to a different process.
Option1:
exec or spawn a child process and execute the load() function in that spawned node app. This is okay if the interval fired is for every 20000 ms as by the time another one fired, the 10sec process will be completed.
Otherwise it is dangerous as it can spawn too many node applications eating up your Systems resources
Option2:
I dont know how much data self.load() accepts and returns. If it is trivial and network overhead is acceptable, make that task a load-balanced web service (may be 4 webservers running in parallel) which accepts (rather point to) 1M records and returns back filtered records.
NOTE
It looks like you are using node async parallel function. But keep a note of this description from the documentation.
Note: parallel is about kicking-off I/O tasks in parallel, not about parallel execution of code. If your tasks do not use any timers or perform any I/O, they will actually be executed in series. Any synchronous setup sections for each task will happen one after the other. JavaScript remains single-threaded.
Related
Assume makeBurger() will take 10 seconds
In synchronous program,
function serveBurger() {
makeBurger();
makeBurger();
console.log("READY") // Assume takes 5 seconds to log.
}
This will take a total of 25 seconds to execute.
So for NodeJs lets say we make an async version of makeBurgerAsync() which also takes 10 seconds.
function serveBurger() {
makeBurgerAsync(function(count) {
});
makeBurgerAsync(function(count) {
});
console.log("READY") // Assume takes 5 seconds to log.
}
Since it is a single thread. I have troubling imagine what is really going on behind the scene.
So for sure when the function run, both async functions will enter event loops and console.log("READY") will get executed straight away.
But while console.log("READY") is executing, no work is really done for both async function right? Since single thread is hogging console.log for 5 seconds.
After console.log is done. CPU will have time to switch between both async so that it can run a bit of each function each time.
So according to this, the function doesn't necessarily result in faster execution, async is probably slower due to switching between event loop? I imagine that, at the end of the day, everything will be spread on a single thread which will be the same thing as synchronous version?
I am probably missing some very big concept so please let me know. Thanks.
EDIT
It makes sense if the asynchronous operations are like query DB etc. Basically nodejs will just say "Hey DB handle this for me while I'll do something else". However, the case I am not understanding is the self-defined callback function within nodejs itself.
EDIT2
function makeBurger() {
var count = 0;
count++; // 1 time
...
count++; // 999999 times
return count;
}
function makeBurgerAsync(callback) {
var count = 0;
count++; // 1 time
...
count++; // 999999 times
callback(count);
}
In node.js, all asynchronous operations accomplish their tasks outside of the node.js Javascript single thread. They either use a native code thread (such as disk I/O in node.js) or they don't use a thread at all (such as event driven networking or timers).
You can't take a synchronous operation written entirely in node.js Javascript and magically make it asynchronous. An asynchronous operation is asynchronous because it calls some function that is implemented in native code and written in a way to actually be asynchronous. So, to make something asynchronous, it has to be specifically written to use lower level operations that are themselves asynchronous with an asynchronous native code implementation.
These out-of-band operations, then communicate with the main node.js Javascript thread via the event queue. When one of these asynchronous operations completes, it adds an event to the Javascript event queue and then when the single node.js thread finishes what it is currently doing, it grabs the next event from the event queue and calls the callback associated with that event.
Thus, you can have multiple asynchronous operations running in parallel. And running 3 operations in parallel will usually have a shorter end-to-end running time than running those same 3 operations in sequence.
Let's examine a real-world async situation rather than your pseudo-code:
function doSomething() {
fs.readFile(fname, function(err, data) {
console.log("file read");
});
setTimeout(function() {
console.log("timer fired");
}, 100);
http.get(someUrl, function(err, response, body) {
console.log("http get finished");
});
console.log("READY");
}
doSomething();
console.log("AFTER");
Here's what happens step-by-step:
fs.readFile() is initiated. Since node.js implements file I/O using a thread pool, this operation is passed off to a thread in node.js and it will run there in a separate thread.
Without waiting for fs.readFile() to finish, setTimeout() is called. This uses a timer sub-system in libuv (the cross platform library that node.js is built on). This is also non-blocking so the timer is registered and then execution continues.
http.get() is called. This will send the desired http request and then immediately return to further execution.
console.log("READY") will run.
The three asynchronous operations will complete in an indeterminate order (whichever one completes it's operation first will be done first). For purposes of this discussion, let's say the setTimeout() finishes first. When it finishes, some internals in node.js will insert an event in the event queue with the timer event and the registered callback. When the node.js main JS thread is done executing any other JS, it will grab the next event from the event queue and call the callback associated with it.
For purposes of this description, let's say that while that timer callback is executing, the fs.readFile() operation finishes. Using it's own thread, it will insert an event in the node.js event queue.
Now the setTimeout() callback finishes. At that point, the JS interpreter checks to see if there are any other events in the event queue. The fs.readfile() event is in the queue so it grabs that and calls the callback associated with that. That callback executes and finishes.
Some time later, the http.get() operation finishes. Internal to node.js, an event is added to the event queue. Since there is nothing else in the event queue and the JS interpreter is not currently executing, that event can immediately be serviced and the callback for the http.get() can get called.
Per the above sequence of events, you would see this in the console:
READY
AFTER
timer fired
file read
http get finished
Keep in mind that the order of the last three lines here is indeterminate (it's just based on unpredictable execution speed) so that precise order here is just an example. If you needed those to be executed in a specific order or needed to know when all three were done, then you would have to add additional code in order to track that.
Since it appears you are trying to make code run faster by making something asynchronous that isn't currently asynchronous, let me repeat. You can't take a synchronous operation written entirely in Javascript and "make it asynchronous". You'd have to rewrite it from scratch to use fundamentally different asynchronous lower level operations or you'd have to pass it off to some other process to execute and then get notified when it was done (using worker processes or external processes or native code plugins or something like that).
I have a server-client based NODE.JS application.
server.js
...
socket.on('message', function(message) {
if(message.code == 103)
{
process_some_data()
}
else
{
console.log("UNKNOWN MESSAGE");
}
});
...
client.js
.. sending responses back to server.js
the process_some_data() function takes about 4 seconds to complete, and when i have just one client it is not a problem, but if i have 10, they all choke and wait till the the last finishes.
I found out that the entire socket event waits till he finishes the current job, for example if i comment process_some_data(), it will not be frozen
I have tried 2 tweaks but the didn't worked :
...
socket.on('message', function(message) {
if(message.code == 103)
{
setTimeout(function() {
process_some_data();
console.log("FINISH");
}, 1)
}
else
{
console.log("UNKNOWN MESSAGE");
}
});
...
And even used http://caolan.github.io/async/ ,but no use :
...
socket.on('message', function(message) {
if(message.code == 103)
{
// Array to hold async tasks
var asyncTasks = [];
async.series([
setTimeout(function() {
process_some_data();
console.log("FINISH");
}, 1)
], function (err, results) {
console.log(results);
});
}
else
{
console.log("UNKNOWN MESSAGE");
}
});
...
How to make this ASYNC? Really need this.
Thank you.
You need multiple processes to solve this with Javascript, because Javascript engines are single-threaded.
What?
When it comes to handling I/O events, such as reading a socket, writing to a file or waiting for a signal, Javascript engines give the appearance of doing multiple things at the same time.
They are actually not: it's just that, under most conditions, processing these events takes so little computation, and the events themselves occur with so much time in between (a microsecond is an eternity for a CPU), that the engine can just process them one after another with plenty of time to spare.
In human time-scale, it looks like the engine is doing a lot of stuff in parallel, but it's just working serially at great speed.
No matter how you schedule your code to run, using setTimeout or Promise, it will still block other events from being processed during the time it's actively computing. Long-running computations (in the scale of seconds, instead of milliseconds) expose the single-threaded nature of the engine: it cannot actually do multiple things at the same time.
Multiple processes
Your computer, however, probably has multiple CPU cores. Unlike the Javascript engine, your hardware is capable of tackling multiple tasks at the same time, at least 1 per core. Even with a single core, your operating system can solve the problem if you run multiple processes.
Since a single Javascript process is single-threaded, you need multiple Javascript processes for this. An easy and time-proven architecture to solve your problem is this:
One Javascript program, running in one process, reads from the socket. Instead of calling process_some_data(), however, it puts all incoming messages in a queue.
This program then sends items from the queue to another Javascript program, running in a different process, that performs the computation using another CPU core. There may be multiple copies of this second process. In a modern computer, it makes sense to have twice as many active processes as you have CPU cores.
A simple approach for Node is to write an HTTP server, using express, that runs the computationally-intensive task. The main program can then use HTTP to delegate tasks to the workers, while still being able to read from the socket.
This is a good article on the topic of multi-processing with Node, using the cluster API.
In the context of Server Side Javascript engines, what is non-blocking I/O or asynchronous I/O? I see this being mentioned as an advantage over Java server side implementations.
Synchronous vs Asynchronous
Synchronous execution usually refers to code executing in sequence. Asynchronous execution refers to execution that doesn't run in the sequence it appears in the code. In the following example, the synchronous operation causes the alerts to fire in sequence. In the async operation, while alert(2) appears to execute second, it doesn't.
Synchronous: 1,2,3
alert(1);
alert(2);
alert(3);
Asynchronous: 1,3,2
alert(1);
setTimeout(() => alert(2), 0);
alert(3);
Blocking vs Non-blocking
Blocking refers to operations that block further execution until that operation finishes. Non-blocking refers to code that doesn't block execution. In the given example, localStorage is a blocking operation as it stalls execution to read. On the other hand, fetch is a non-blocking operation as it does not stall alert(3) from execution.
// Blocking: 1,... 2
alert(1);
var value = localStorage.getItem('foo');
alert(2);
// Non-blocking: 1, 3,... 2
alert(1);
fetch('example.com').then(() => alert(2));
alert(3);
Advantages
One advantage of non-blocking, asynchronous operations is that you can maximize the usage of a single CPU as well as memory.
Synchronous, blocking example
An example of synchronous, blocking operations is how some web servers like ones in Java or PHP handle IO or network requests. If your code reads from a file or the database, your code "blocks" everything after it from executing. In that period, your machine is holding onto memory and processing time for a thread that isn't doing anything.
In order to cater other requests while that thread has stalled depends on your software. What most server software do is spawn more threads to cater the additional requests. This requires more memory consumed and more processing.
Asynchronous, non-blocking example
Asynchronous, non-blocking servers - like ones made in Node - only use one thread to service all requests. This means an instance of Node makes the most out of a single thread. The creators designed it with the premise that the I/O and network operations are the bottleneck.
When requests arrive at the server, they are serviced one at a time. However, when the code serviced needs to query the DB for example, it sends the callback to a second queue and the main thread will continue running (it doesn't wait). Now when the DB operation completes and returns, the corresponding callback pulled out of the second queue and queued in a third queue where they are pending execution. When the engine gets a chance to execute something else (like when the execution stack is emptied), it picks up a callback from the third queue and executes it.
var startTime = new Date().getTime();
var getEndTime = () => {
var tempEndTime = new Date().getTime();
var second = (tempEndTime - startTime)/1000
return `took ${second} sec...to finish\n`
}
console.log('1: start App', getEndTime())
setTimeout(()=>{
console.log('2: setTimeout', getEndTime())
}, 1000)
console.log('3: End App', getEndTime())
// console -> Process Order: 1 -> 3 -> 2
In the context of Server Side Javascript engines, what is non-blocking I/O or asynchronous I/O? I see this being mentioned as an advantage over Java server side implementations.
Synchronous vs Asynchronous
Synchronous execution usually refers to code executing in sequence. Asynchronous execution refers to execution that doesn't run in the sequence it appears in the code. In the following example, the synchronous operation causes the alerts to fire in sequence. In the async operation, while alert(2) appears to execute second, it doesn't.
Synchronous: 1,2,3
alert(1);
alert(2);
alert(3);
Asynchronous: 1,3,2
alert(1);
setTimeout(() => alert(2), 0);
alert(3);
Blocking vs Non-blocking
Blocking refers to operations that block further execution until that operation finishes. Non-blocking refers to code that doesn't block execution. In the given example, localStorage is a blocking operation as it stalls execution to read. On the other hand, fetch is a non-blocking operation as it does not stall alert(3) from execution.
// Blocking: 1,... 2
alert(1);
var value = localStorage.getItem('foo');
alert(2);
// Non-blocking: 1, 3,... 2
alert(1);
fetch('example.com').then(() => alert(2));
alert(3);
Advantages
One advantage of non-blocking, asynchronous operations is that you can maximize the usage of a single CPU as well as memory.
Synchronous, blocking example
An example of synchronous, blocking operations is how some web servers like ones in Java or PHP handle IO or network requests. If your code reads from a file or the database, your code "blocks" everything after it from executing. In that period, your machine is holding onto memory and processing time for a thread that isn't doing anything.
In order to cater other requests while that thread has stalled depends on your software. What most server software do is spawn more threads to cater the additional requests. This requires more memory consumed and more processing.
Asynchronous, non-blocking example
Asynchronous, non-blocking servers - like ones made in Node - only use one thread to service all requests. This means an instance of Node makes the most out of a single thread. The creators designed it with the premise that the I/O and network operations are the bottleneck.
When requests arrive at the server, they are serviced one at a time. However, when the code serviced needs to query the DB for example, it sends the callback to a second queue and the main thread will continue running (it doesn't wait). Now when the DB operation completes and returns, the corresponding callback pulled out of the second queue and queued in a third queue where they are pending execution. When the engine gets a chance to execute something else (like when the execution stack is emptied), it picks up a callback from the third queue and executes it.
var startTime = new Date().getTime();
var getEndTime = () => {
var tempEndTime = new Date().getTime();
var second = (tempEndTime - startTime)/1000
return `took ${second} sec...to finish\n`
}
console.log('1: start App', getEndTime())
setTimeout(()=>{
console.log('2: setTimeout', getEndTime())
}, 1000)
console.log('3: End App', getEndTime())
// console -> Process Order: 1 -> 3 -> 2
With Node.js, or eventlet or any other non-blocking server, what happens when a given request takes long, does it then block all other requests?
Example, a request comes in, and takes 200ms to compute, this will block other requests since e.g. nodejs uses a single thread.
Meaning your 15K per second will go down substantially because of the actual time it takes to compute the response for a given request.
But this just seems wrong to me, so I'm asking what really happens as I can't imagine that is how things work.
Whether or not it "blocks" is dependent on your definition of "block". Typically block means that your CPU is essentially idle, but the current thread isn't able to do anything with it because it is waiting for I/O or the like. That sort of thing doesn't tend to happen in node.js unless you use the non-recommended synchronous I/O functions. Instead, functions return quickly, and when the I/O task they started complete, your callback gets called and you take it from there. In the interim, other requests can be processed.
If you are doing something computation-heavy in node, nothing else is going to be able to use the CPU until it is done, but for a very different reason: the CPU is actually busy. Typically this is not what people mean when they say "blocking", instead, it's just a long computation.
200ms is a long time for something to take if it doesn't involve I/O and is purely doing computation. That's probably not the sort of thing you should be doing in node, to be honest. A solution more in the spirit of node would be to have that sort of number crunching happen in another (non-javascript) program that is called by node, and that calls your callback when complete. Assuming you have a multi-core machine (or the other program is running on a different machine), node can continue to respond to requests while the other program crunches away.
There are cases where a cluster (as others have mentioned) might help, but I doubt yours is really one of those. Clusters really are made for when you have lots and lots of little requests that together are more than a single core of the CPU can handle, not for the case where you have single requests that take hundreds of milliseconds each.
Everything in node.js runs in parallel internally. However, your own code runs strictly serially. If you sleep for a second in node.js, the server sleeps for a second. It's not suitable for requests that require a lot of computation. I/O is parallel, and your code does I/O through callbacks (so your code is not running while waiting for the I/O).
On most modern platforms, node.js does us threads for I/O. It uses libev, which uses threads where that works best on the platform.
You are exactly correct. Nodejs developers must be aware of that or their applications will be completely non-performant, if long running code is not asynchronous.
Everything that is going to take a 'long time' needs to be done asynchronously.
This is basically true, at least if you don't use the new cluster feature that balances incoming connections between multiple, automatically spawned workers. However, if you do use it, most other requests will still complete quickly.
Edit: Workers are processes.
You can think of the event loop as 10 people waiting in line to pay their bills. If somebody is taking too much time to pay his bill (thus blocking the event loop), the other people will just have to hang around waiting for their turn to come.. and waiting...
In other words:
Since the event loop is running on a single thread, it is very
important that we do not block it’s execution by doing heavy
computations in callback functions or synchronous I/O. Going over a
large collection of values/objects or performing time-consuming
computations in a callback function prevents the event loop from
further processing other events in the queue.
Here is some code to actually see the blocking / non-blocking in action:
With this example (long CPU-computing task, non I/O):
var net = require('net');
handler = function(req, res) {
console.log('hello');
for (i = 0; i < 10000000000; i++) { a = i + 5; }
}
net.createServer(handler).listen(80);
if you do 2 requests in the browser, only a single hello will be displayed in the server console, meaning that the second request cannot be processed because the first one blocks the Node.js thread.
If we do an I/O task instead (write 2 GB of data on disk, it took a few seconds during my test, even on a SSD):
http = require('http');
fs = require('fs');
buffer = Buffer.alloc(2*1000*1000*1000);
first = true;
done = false;
write = function() {
fs.writeFile('big.bin', buffer, function() { done = true; });
}
handler = function(req, res) {
if (first) {
first = false;
res.end('Starting write..')
write();
return;
}
if (done) {
res.end("write done.");
} else {
res.end('writing ongoing.');
}
}
http.createServer(handler).listen(80);
here we can see that the a-few-second-long-IO-writing-task write is non-blocking: if you do other requests in the meantime, you will see writing ongoing.! This confirms the well-known non-blocking-for-IO features of Node.js.