There is a lot of emitting and listening for events in node.js.
I'm trying to decide if I should emit events inline and then return from a function or if I should generally use process.nextTick to emit events causing them to run later.
Which is the right (best) way to do this generally?
Someone told me that node.js's built in modules mostly use process.nextTick.
Generally in node v0.10 and newer you should use setImmediate. I cover all the various options in my blog post setTimeout and friends. The main point to note with event emitters is you need to allow the caller at least one tick to bind their event listeners before you start emitting events. This pattern is normally considered OK and should be supported:
var myThing = new Thing();
//if myThing immediately emits events,
// they will be missed because they aren't bound yet
myThing.on('event', handleEvent);
//If events are delayed at least one tick, all is well,
// the listener is bound and all events are handled
For node earlier that v0.10, process.nextTick should be used.
Note that if your emitter's logic requires async calls naturally (because it does I/O), you don't need setImmediate as doing the I/O already allows the event loop to complete the tick. You only need setImmediate in cases where you sometimes need to do I/O but in this case you don't need it, for example, you have a cached result from previous I/O.
This is a great article that explains nextTick() and gives some good reasons why events should generally be emited in a nextTick().
Related
In my program, I have a few timer(1000).subscribe() instances as well as some timer(1000, 1000).subscribe() parts.
I was experiencing some memory leak issues and was wondering if I could alleviate those by unsubscribing from timers. Unsubscribing from recurring timers seems straight forward and necessary, but do I also have to unsubscribe from timers that only emit once?
The second part of my question is if there is a better way to unsubscribe from the emitting timer than to put it into a variable like so:
const myTimer = timer(1000).subscribe(() => {
myTimer.unsubscribe();
});
Thanks!
timer will complete itself after one emission, so there's no need for unsubscribe
For the alternative way to unsubscribe you can use the subscription instance. By using that, you benefit from storing all subscriptions and unsubscribe all with one call.
subs=new Subscription();
subs.add(timer(1000).subscribe())
subs.unsubscribe()
I am doing an IO wait operation inside a for loop now the thing is when all of the operations terminates I want to send the response to the server. Now I was just wondering that suppose two IO operation terminates exactly at the same time now can they execute code at the same time(parallel) or will they execute serially?
As far as I know, as Node is Concurrent but not the Parallel language so I don't think they will execute at the same time.
node.js runs Javascript with a single thread. That means that two pieces of Javascript can never be running at the exact same moment.
node.js processes I/O completion using an event queue. That means when an I/O operation completes, it places an event in the event queue and when that event gets to the front of the event queue and the JS interpreter has finished whatever else it was doing, then it will pull that event from the event queue and call the callback associated with it.
Because of this process, even if two I/O operations finish at basically the same moment, one of them will put its completion event into the event queue before the other (access to the event queue internally is likely controlled by a mutex so one will get the mutex before the other) and that one's completion callback will get into the event queue first and then called before the other. The two completion callbacks will not run at the exact same time.
Keep in mind that more than one piece of Javascript can be "in flight" or "in process" at the same time if it contains non-blocking I/O operations or other asynchronous operations. This is because when you "wait" for an asynchronous operation to complete in Javscript, you return control back to the system and you then resume processing only when your completion callback is called. While the JS interpreter is waiting for an asynchronous I/O operation to complete and the associated callback to be called, then other Javascript can run. But, there's still only one piece of Javascript actually ever running at a time.
As far as I know, as Node is Concurrent but not the Parallel language so I don't think they will execute at the same time.
Yes, that's correct. That's not exactly how I'd describe it since "concurrent" and "parallel" don't have strict technical definitions, but based on what I think you mean by them, that is correct.
you can use Promise.all :
let promises = [];
for(...)
{
promises.push(somePromise); // somePromise represents your IO operation
}
Promise.all(promises).then((results) => { // here you send the response }
You don't have to worry about the execution order.
Node.js is designed to be single thread. So basically there is no way that 'two IO operation terminates exactly at the same time' could happen. They will just finish one by one.
I'm working on a project where I have a Feeds page which shows all type of post by all users. Type specific list page and Posts' detail page.
Pages-
1. Feeds
2. List (Type specific)
3. Detail (detail of a post)
So I have following Mongo collection -
1. Feed
2. type1 post
3. type2 post
4. type3...
Now when a user post a new Post I save it to respective collection lets say to 'type1 post' and return success to browser. But I also want to update my 'Feed' collection with same data. I don't want it to be done before response is send. Because that will increase User's wait time. Hence I have used Events. Here's my code -
const emitter = new event.EventEmitter();
function savePost(){
// Code to save data to Mongo collection
emitter.emit('addToFeeds', data);
console.log('emit done');
return res.json(data);
}
emitter.on('addToFeeds', function(data){
// code to save data to Feeds collection
console.log('emitter msg - ', data);
});
Now when I check the console.log output, it shows "emitter msg -" first and then "emit done". That's why I'm assuming emitter.on code is executing before res.json(data);
Now I'm wondering does Events are blocking code? If I have to update Feeds in background or after response is sent what is the right way? In future I also want to implement caching so I also have to update cache when even a post is added, that too I want to do after response is sent or in background.
Yes, events are synchronous and blocking. They are implemented with simple function calls. If you look at the eventEmitter code, to send an event to all listeners, it literally just iterates through an array of listeners and calls each listener callback, one after the other.
Now I'm wondering does Events are blocking code?
Yes. In the doc for .emit(), it says this: "Synchronously calls each of the listeners registered for the event named eventName, in the order they were registered, passing the supplied arguments to each."
And, further info in the doc in this section Asynchronous vs. Synchronous where it says this:
The EventEmitter calls all listeners synchronously in the order in which they were registered. This is important to ensure the proper sequencing of events and to avoid race conditions or logic errors. When appropriate, listener functions can switch to an asynchronous mode of operation using the setImmediate() or process.nextTick() methods:
If I have to update Feeds in background or after response is sent what is the right way?
Your eventListener can schedule when it wants to actually execute its code with a setTimeout() or a setImmediate() or process.nextTick() if it wants the other listeners and other synchronous code to finish running before it does its work. So, you register a normal listener (which will get called synchronously) and then inside that, you can use a setTimeout() or setImmediate() or process.nextTick() and put the actual work inside that callback. This will delay running your code until after the current Javascript that triggered the initial event is done running.
There is no actual "background processing" in node.js for pure Javascript code. node.js is single threaded so while you're running some Javascript, no other Javascript can run. Actual background processing would have to be done either with existing asynchronous operations (that use native code to run things in the background) such as network I/O or disk I/O) or by running another process to do the work (that other process can be any type of code including another node.js process).
Events are synchronous and will block. This is done so that you can bind events in a specific order and cascade them in that order. You can make each item asynchronous, and if you're making HTTP requests at those events, that's going to happen async, but the events themselves are started synchronously.
See: https://nodejs.org/api/events.html#events_emitter_emit_eventname_args
And: https://nodejs.org/api/events.html#events_asynchronous_vs_synchronous
Using ExpressJS and Socket.IO I have an HTML scene where multiple users can connect to NodeJS. I am about to do some animation that has to sync to all clients.
In the client, I know animation can be achieved by setInterval() (not time-ideal) then socket.emit() to NodeJS. But is there an Idle loop in NodeJS that can be used for master-controlling animations and io.sockets.emit() to update everyone about everyone?
EDIT: I want to do general "animation" of values in node.js e.g. pseudocode:
process.idle(function() {
// ...
itempos.x += (itempos.dest - itempos.x) / 20; // easing
itempos.y += (itempos.dest - itempos.y) / 20; // easing
io.sockets.broadcast('update', itempos);
// ...
});
Being a server-side framework it will rarely idle (CPU or I/O). Besides idleloop is more suited for DOM requirements. But in node.js you have the following functions:
process.nextTick : Execute callback after current event queue finishes i.e. at the beginning of next event loop. It does not allow I/O execution until maxTickDepth nextTick calls are executed. If used too much it can prevent I/O from occurring.
setImmediate : Execute callback after I/O callbacks in current event loop are finished. Allows I/O to happen between multiple setImmediate calls.
Given what you want setImmediate is more suited for your needs.
Check out the Timers docs: http://nodejs.org/api/timers.html
All of the timer functions are globals.
setInterval(callback, delay, [arg], [...])
To schedule the repeated execution of callback every delay milliseconds. Returns a intervalId for possible use with clearInterval(). Optionally you can also pass arguments to the callback.
For synchronized client animation it may make sense to do sequences in chunks at a slower rate than trying to squeeze as many websocket emissions into the animation duration. Human eyes are much slower than websockets in my experience.
There's tons of client frameworks that will do the easing for you, not a server concern.
(All of this oblivious to your use case, of course!)
Does asynchronous call always create a new thread?
Example:
If JavaScript is single threaded then how can it do an async postback? Is it actually blocking until it gets a callback? If so, is this really an async call?
This is an interesting question.
Asynchronous programming is a paradigm of programming that is principally single threaded, i.e. "following one thread of continuous execution".
You refer to javascript, so lets discuss that language, in the environment of a web browser. A web browser runs a single thread of javascript execution in each window, it handles events (such as onclick="someFunction()") and network connections (such as xmlhttprequest calls).
<script>
function performRequest() {
xmlhttp.open("GET", "someurl", true);
xmlhttp.onreadystatechange = function() {
if (xmlhttp.readyState == 4) {
alert(xmlhttp.responseText);
}
}
xmlhttp.send(sometext);
}
</script>
<span onclick="performRequest()">perform request</span>
(This is a nonworking example, for demonstration of concepts only).
In order to do everything in an asynchronous manner, the controlling thread has what is known as a 'main loop'. A main loop looks kind of like this:
while (true) {
event = nextEvent(all_event_sources);
handler = findEventHandler(event);
handler(event);
}
It is important to note that this is not a 'busy loop'. This is kind of like a sleeping thread, waiting for activity to occur. Activity could be input from the user (Mouse Movement, a Button Click, Typing), or it could be network activity (The response from the server).
So in the example above,
When the user clicks on the span, a ButtonClicked event would be generated, findEventHandler() would find the onclick event on the span tag, and then that handler would be called with the event.
When the xmlhttp request is created, it is added to the all_event_sources list of event sources.
After the performRequest() function returns, the mainloop is waiting at the nextEvent() step waiting for a response. At this point there is nothing 'blocking' further events from being handled.
The data comes back from the remote server, nextEvent() returns the network event, the event handler is found to be the onreadystatechange() method, that method is called, and an alert() dialog fires up.
It is worth noting that alert() is a blocking dialog. While that dialog is up, no further events can be processed. It's an eccentricity of the javascript model of web pages that we have a readily available method that will block further execution within the context of that page.
The Javascript model is single-threaded. An asynchronous call is not a new thread, but rather interrupts an existing thread. It's analogous to interrupts in a kernel.
Yes it makes sense to have asynchronous calls with a single thread. Here's how to think about it: When you call a function within a single thread, the state for the current method is pushed onto a stack (i.e. local variables). The subroutine is invoked and eventually returns, at which time the original state is popped off the stack.
With an asynchronous callback, the same thing happens! The difference is that the subroutine is invoked by the system, not by the current code invoking a subroutine.
A couple notes about JavaScript in particular:
XMLHttpRequests are non-blocking by default. The send() method returns immediately after the request has been relayed to the underlying network stack. A response from the server will schedule an invocation of your callback on the event loop as discussed by the other excellent answers.
This does not require a new thread. The underlying socket API is selectable, similar to java.nio.channels in Java.
It's possible to construct synchronous XMLHttpRequest objects by passing false as the third parameter to open(). This will cause the send() method to block until a response has been received from the server, thus placing the event loop at the mercy of network latency and potentially hanging the browser until network timeout. This is a Bad Thing™.
Firefox 3.5 will introduce honest-to-god multithreaded JavaScript with the Worker class. The background code runs in a completely separate environment and communicates with the browser window by scheduling callbacks on the event loop.
In many GUI applications, an async call (like Java's invokeLater) merely adds the Runnable object to its GUI thread queue. The GUI thread is already created, and it doesn't create a new thread. But threads aren't even strictly required for an asynchronous system. Take, for example, libevent, which uses select/poll/kqueue, etc. to make non-blocking calls to sockets, which then fires callbacks to your code, completely without threads.
No, but more than one thread will be involved.
An asynchronous call might launch another thread to do the work, or it might post a message into a queue on another, already running thread. The caller continues and the callee calls back once it processes the message.
If you wanted to do a synchronous call in this context, you'd need to post a message and actively wait for the callback to happen.
So in summary: More than one thread will be involved, but it doesn't necessarily create a new thread.
I don't know about javascript, but for instance in the Windows Forms world, asynchronous invocations can be made without multiple threads. This has to do with the way the Windows Message Pump operates. Basically a Windows Forms application sets up a message queue through which Windows places messages notifying it about events. For instance, if you move the mouse, messages will be placed on that queue. The Windows Forms application will be in an endless loop consuming all the messages that are thrown at it. According to what each message contains it will move windows around, repaint them or even invoke user-defined methods, amongst other things. Calls to methods are identified by delegates. When the application finds a delegate instance in the queue, it happily invokes the method referred by the delegate.
So, if you are in a method doing something and want to spawn some asynchronous work without creating a new thread, all you have to do is place a delegate instance into the queue, using the Control.BeginInvoke method. Now, this isn't actually multithreaded, but if you throw very small pieces of work to the queue, it will look like multithreaded. If, on the other hand you give it a time consuming method to execute, the application will freeze until the method is done, which will look like a jammed application, even though it is doing something.