Does Go have an equivalent of node.js' "emitter"?
I'm teaching myself Go by porting over a node.js library I wrote. In the node version, the library emits an event once something happens (e.g. it listens on UDP port 1234 and when "ABC" is received, "abcreceived" is emitted so the calling code can respond as necessary (e.g. sending back "DEF")
I've seen channels in Go (and am currently reading up on them), but as I'm still new to this language, I don't know if (or how, for that matter) that can be used to communicate with whatever code is using my library.
I've also seen https://github.com/chuckpreslar/emission, but am not sure if this is acceptable, or if there's a better ("Best practice") way of doing things.
Go and Node.js are very different. Node.js supports concurrency only via callbacks. There might be various ways of dressing them up, but they're fundamentally callbacks.
In Node.js, there is no parallelism; Node.js has a single-threaded runtime. When Node.js async is used to achieve what is called 'parallel' execution, it isn't parallel in the sense used in Go, but concurrent.
Concurrency is not parallelism in the Go world.
Go has explicit concurrency based on Communicating Sequential Processes (CSP), a mathematical basis conceived by Tony Hoare at Oxford. The runtime interleaves cooperating processes called goroutines by time-slicing them onto the available CPU cores. Within each goroutine, the code is single threaded, so is easy to write. In the simple case, no data is shared between goroutines; instead messages pass between them along channels. In this way, there is no need for callbacks.
When goroutines get blocked waiting for I/O, that's OK because they don't use any CPU time until they're unblocked. Their memory footprint is slight and you can have very large numbers of them. So callbacks are not needed for I/O operations either.
Because the execution models of Go and Node.js are about as different as they could be, attempting to port code from one to the other is very likely to lead to very clumsy solutions. It's better to start from the original requirements and implement from scratch.
It would be possible to distort the Go concurrency model using function arguments to behave like callbacks. This would be a bad idea because it would not be idiomatic and would lose the benefits that CSP gives.
So by reading others' Go code and some links in the comments to my question, I think channels are the way to go.
In my library code (semi pseudo-code):
// Make a new channel called "Events"
var Events = make(chan
func doSomething() {
// ...
Events <-"abcreceived" // Add "abcreceived" to the Events channel
}
And in the code that will use my library:
evt := <-mylib.Events
switch evt {
case "abcreceived":
sendBackDEF()
break
// ...
}
I still prefer node.js' EventEmitter (because you can transfer data back easily) but for simple things, this should suffice.
Related
Many articles online demonstrates nodejs as an example of reactor pattern. Isn't it rather proactor?
As far as I understand, the difference between the two is:
reactor handles events in a single thread (synchronously),
proactor handles events is multiple threads (asynchronously) with completion callbacks.
For example in this article:
Reactor Pattern is an idea of non-blocking I/O operations in Node.js. This pattern provides a handler(in case of Node.js, a callback function) that is associated with each I/O operation. When an I/O request is generated, it is submitted to a demultiplexer.
Isn't it actually definition of proactor?
I wasn't familiar with the Proactor design pattern. After reading a bit about it I think I understand your confusion.
Many articles online demonstrates nodejs as an example of reactor pattern
This is true.
Isn't it actually definition of proactor?
This is also true.
The difference is your point of view.
Internally, node's event loop is a blocking call (ironically). That's just the most efficient way to use non-blocking I/O. Different OSes have different functions to request the OS to wake your process up if something you are interested in happens. Due to POSIX requirements there is a cross-platform API that all modern OSes support: select(). Node.js actually uses libuv which automatically picks the right API at compile time depending on the target platform. But for the purposes of this answer we're going to focus on select(). So lets look at select():
numberOfEvents = select(numberOfWaits, read, write, err, timeout);
The select() function blocks for up to timeout milliseconds or something happens to either the read, write or err files/sockets. With just a single function the OS provides enough functionality to implement most of node.js from timers like setTimeout() and setInterval() to listening to network sockets. Using select() the event loop looks something like this:
// Pseudocode:
while(1) {
evaluateJavascript();
timeout = calculateTimers();
events = select(n, read, write, err, timeout);
if (events > 0 || timersActive()) {
getCallbacks(events, read, write, err, timers());
}
}
This is basically a Reactor design pattern.
However, node hides this away in its implementation. What it exposes to Javascript programmers is a set of APIs that registers callbacks and calls those callbacks when an event happens. This is partly historical (the browser APIs was designed that way) and partly practical (it's a much more flexible architecture - almost all GUI frameworks from GTK to wxWindows to .Net works this way).
You may recognise that this sounds a lot like a Proactor design pattern. And in fact it is.
So node.js itself is an example of Reactor design pattern.
Javascript programs written in node.js are examples of Proactor design pattern.
The distinction has nothing to do with multithreading. It is as follows:
Reactor - I want to read from a socket, so I subscribe to a data-is-available kind of event and, and when it fires react to it by synchronously reading the available amount.
Proactor - I want to read from a socket, so I initiate a reading operation (one that proactively reads the data, without waiting for me to react to it's availability), and subscribe to some kind of read-is-complete notification, wherein the read data is immediately available to me.
Node has both kinds of APIs, e.g. stream.ReadableStream#readable/stream.ReadableStream#read are a Reactor interface, while
fs.readFile is a Proactor.
I've been reading a lot about the Event Loop, and I understand the abstraction provided whereby I can make an I/O request (let's use fs.readFile(foo.txt)) and just pass in a callback that will be executed once a particular event indicates completion of the file reading is fired. However, what I do not understand is where the function that is doing the work of actually reading the file is being executed. Javascript is single-threaded, but there are two things happening at once: the execution of my node.js file and of some program/function actually reading data from the hard drive. Where does this second function take place in relation to node?
The Node event loop is truly single threaded. When we start up a program with Node, a single instance of the event loop is created and placed into one thread.
However for some standard library function calls, the node C++ side and libuv decide to do expensive calculations outside of the event loop entirely. So they will not block the main loop or event loop. Instead they make use of something called a thread pool that thread pool is a series of (by default) four threads that can be used for running computationally intensive tasks. There are ONLY FOUR things that use this thread pool - DNS lookup, fs, crypto and zlib. Everything else execute in the main thread.
"Of course, on the backend, there are threads and processes for DB access and process execution. However, these are not explicitly exposed to your code, so you can’t worry about them other than by knowing that I/O interactions e.g. with the database, or with other processes will be asynchronous from the perspective of each request since the results from those threads are returned via the event loop to your code. Compared to the Apache model, there are a lot less threads and thread overhead, since threads aren’t needed for each connection; just when you absolutely positively must have something else running in parallel and even then the management is handled by Node.js." via http://blog.mixu.net/2011/02/01/understanding-the-node-js-event-loop/
Its like using, setTimeout(function(){/*file reading code here*/},1000);. JavaScript can run multiple things side by side like, having three setInterval(function(){/*code to execute*/},1000);. So in a way, JavaScript is multi-threading. And for actually reading from/or writing to the hard drive, in NodeJS, if you use:
var child=require("child_process");
function put_text(file,text){
child.exec("echo "+text+">"+file);
}
function get_text(file){
//JQuery code for getting file contents here (i think)
return JQueryResults;
}
These can also be used for reading and writing to/from the hard drive using NodeJS.
I've been doing a fair amount of work with Node lately, trying to build a system which has certain characteristics, one of which is non-blocking / parallelism - a Node strong suit, as I understand it.
What I don't fully understand is when a separate thread is spun off to handle some processing. I'm pretty sue this happens on a function call/call back, but certainly not all of them.
In my specific case, it's an Express based app. At app start-up it does several things including instantiating a RabbitMQ based "bus", an object with a method which will write to the bus (objA) and object which will subscribe to the bus and process messages coming across it (objB).
objA will write to the bus inside an express callback
app.put((req,res) => {
objA.methodWhichWritesToBus();
});
I believe at this point, that objA.methodWhichWritesToBus is executed in a background/worker thread - whatever you call it, not on the main event loop.
Is that the only point at which this sort of thing happens? methodWhichWritesToBus is IO instensive (it calls an elastic search service on another box and brings back 10's to 100's of thousands of records) with lots of chained promises etc., but none of that gets split off, does it?
How about the fact that the obj on which the method is called is instantiated outside the Express callback - does that affect the parallel-ism?
Finally, are the ways to effect/force a method etc to "run in the background"?
I've been noodling this, testing it, for awhile now but all on one machine so it's difficult to tell what's going on.
Who can clarify this for me?
Pre-answer: this is a topic best learned by going and reading, doing coding exercises to solidify your understanding, and working with the technology in a significant way. You're not going to "get it" based on a Q&A format. That said...
What I don't fully understand is when a separate thread is spun off to handle some processing.
Never, sort of. "Processing" as in the computation that happens in your javascript program, happens in the main event loop thread. End of story. However, waiting on I/O to come back from the OS is not considered "processing" so there are various queues managed by node and the OS to track pending I/O requests and invoke callbacks when data is ready. There are a handful of threads node uses internally to manage this stuff with the OS, but from your program's perspective, those threads are irrelevant. Your program can ask node to do some IO, then your program keeps running in parallel, and when the I/O is done, node will eventually invoke the callback in the main event loop and you can process the results.
I believe at this point, that objA.methodWhichWritesToBus is executed in a background/worker thread - whatever you call it, not on the main event loop.
You call it "asynchronously" and it happens whenever you do IO, including filesystem calls, networking, or child processes. Which is to say, quite a lot.
How about the fact that the obj on which the method is called is instantiated outside the Express callback - does that affect the parallel-ism?
Nope.
Finally, are the ways to effect/force a method etc to "run in the background"?
Generally I/O is done asynchronously by default, so no you don't normally need to force anything to run in the background. It's baked into the node design by way of the node core APIs themselves. However, there are ways to delay synchronous processing to a future event loop using setImmediate, setTimeout, or process.nextTick. I explain these in some detail in my blog post setTimeout and friends.
More precisely, all networking is asynchronous. End of story. Specifically, the APIs in node core that are available are all asynchronous, and there's simply no synchronous API available in node. For filesystem IO and child processes, there are both synchronous and asynchronous APIs, but the synchronous APIs must only be used under special limited circumstances, and if you don't know confidently that it's OK in this specific case to make a synchronous IO API call, you should use the asynchronous API so you don't break the lynchpin that makes node perform as it does.
I found many talks saying that Node.js is bad because of callback hell and Go is good because of its synchronous model.
What I feel is Go can also do callback as same as Node.js but in a synchronous way. As we can pass anonymous function and do closure things
So, why are they comparing Go and Node.js in callback perspective as if Go cannot become callback hell.
Or I misunderstand the meaning of callback and anonymous function in Go?
A lot of things take time, e.g. waiting on a network socket, a file system read, a system call, etc. Therefore, a lot of languages, or more precisely their standard library, include asynchronous version of their functions (often in addition to the synchronous version), so that your program is able to do something else in the mean-time.
In node.js things are even more extreme. They use a single-threaded event loop and therefore need to ensure that your program never blocks. They have a very well written standard library that is built around the concept of being asynchronous and they use callbacks in order to notify you when something is ready. The code basically looks like this:
doSomething1(arg1, arg2, function() {
doSomething2(arg1, arg2, function() {
doSomething3(function() {
// done
});
});
});
somethingElse();
doSomething1 might take a long time to execute (because it needs to read from the network for example), but your program can still execute somethingElse in the mean time. After doSomething1 has been executed, you want to call doSomething2 and doSomething3.
Go on the other hand is based around the concept of goroutines and channels (google for "Communicating Sequential Processes", if you want to learn more about the abstract concept). Goroutines are very cheap (you can have several thousands of them running at the same time) and therefore you can use them everywhere. The same code might look like this in Go:
go func() {
doSomething1(arg1, arg2)
doSomething2(arg1, arg2)
doSomething3()
// done
}()
somethingElse()
Whereas node.js focus on providing only asynchronous APIs, Go usually encourages you to write only synchronous APIs (without callbacks or channels). The call to doSomething1 will block the current goroutine and doSomething2 will only be executed after doSomething1 has finished. But that's not a problem in Go, since there are usually other goroutines available that can be scheduled to run on the system thread. In this case, somethingElse is part of another goroutine and can be executed in the meantime, just like in the node.js example.
I personally prefer the Go code, since it's easier to read and reason about. Another advantage of Go is that it also works well with computation heavy tasks. If you start a heavy computation in node.js that doesn't need to wait for network of filesystem calls, this computation basically blocks your event loop. Go's scheduler on the other hand will do its best to dispatch the goroutines on a few number of system threads and the OS might run those threads in parallel if your CPU supports it.
What I feel is Golang can also do callback as same as Node.js but in a synchronous way. As we can pass anonymous function and do closure things
So, why are they comparing Golang and Node.js in callback perspective as if Golang cannot become callback hell.
Yes, of course it is possible to mess things up in Go as well. The reason why you don't see as much callbacks as in node.js is that Go has channels for communication, which allow for a way of structuring your code without using callbacks.
So, since there are channels, callbacks are not used as often therefore it is unlikely to stumble over callback infested code. Of course this doesn't mean that you cannot write scary code with channels as well...
Redis is very fast. For most part on my machine it is as fast as say native Javascript statements or function calls in node.js. It is easy/painless to write regular Javascript code in node.js because no callbacks are needed. I don't see why it should not be that easy to get/set key/value data in Redis using node.js.
Assuming node.js and Redis are on the same machine, are there any npm libraries out there that allow interacting with Redis on node.js using blocking calls? I know this has to be a C/C++ library interfacing with V8.
I suppose you want to ensure all your redis insert operations have been performed. To achieve that, you can use the MULTI commands to insert keys or perform other operations.
The https://github.com/mranney/node_redis module queues up the commands pushed in multi object, and executes them accordingly.
That way you only require one callback, at the end of exec call.
This seems like a common bear-trap for developers who are trying to get used to Node's evented programming model.
What happens is this: you run into a situation where the async/callback pattern isn't a good fit, you figure what you need is some way of doing blocking code, you ask Google/StackExchange about blocking in Node, and all you get is admonishment on how bad blocking is.
They're right - blocking, ("wait for the result of this before doing anything else"), isn't something you should try to do in Node. But what I think is more helpful is to realize that 99.9% of the time, you're not really looking for a way to do blocking, you're just looking for a way to make your app, "wait for the result of this before going on to do that," which is not exactly the same thing.
Try looking into the idea of "flow control" in Node rather than "blocking" for some design patterns that could be a clearer fit for what you're trying to do. Here's a list of libraries to check out:
https://github.com/joyent/node/wiki/modules#wiki-async-flow
I'm new to Node too, but I'm really digging Async: https://github.com/caolan/async
Blocking code creates a MASSIVE bottleneck.
If you use blocking code your server will become INCREDIBLY slow.
Remember, node is single threaded. So any blocking code, will block node for every connected client.
Your own benchmarking shows it's fast enough for one client. Have you benchmarked it with a 1000 clients? If you try this you will see why blocking code is bad
Whilst Redis is quick it is not instantaneous ... this is why you must use a callback if you want to continue execution ensuring your values are there.
The only way I think you could (and am not suggesting you do) achieve this use a callback with a variable that is the predicate for leaving a timer.