I found many talks saying that Node.js is bad because of callback hell and Go is good because of its synchronous model.
What I feel is Go can also do callback as same as Node.js but in a synchronous way. As we can pass anonymous function and do closure things
So, why are they comparing Go and Node.js in callback perspective as if Go cannot become callback hell.
Or I misunderstand the meaning of callback and anonymous function in Go?
A lot of things take time, e.g. waiting on a network socket, a file system read, a system call, etc. Therefore, a lot of languages, or more precisely their standard library, include asynchronous version of their functions (often in addition to the synchronous version), so that your program is able to do something else in the mean-time.
In node.js things are even more extreme. They use a single-threaded event loop and therefore need to ensure that your program never blocks. They have a very well written standard library that is built around the concept of being asynchronous and they use callbacks in order to notify you when something is ready. The code basically looks like this:
doSomething1(arg1, arg2, function() {
doSomething2(arg1, arg2, function() {
doSomething3(function() {
// done
});
});
});
somethingElse();
doSomething1 might take a long time to execute (because it needs to read from the network for example), but your program can still execute somethingElse in the mean time. After doSomething1 has been executed, you want to call doSomething2 and doSomething3.
Go on the other hand is based around the concept of goroutines and channels (google for "Communicating Sequential Processes", if you want to learn more about the abstract concept). Goroutines are very cheap (you can have several thousands of them running at the same time) and therefore you can use them everywhere. The same code might look like this in Go:
go func() {
doSomething1(arg1, arg2)
doSomething2(arg1, arg2)
doSomething3()
// done
}()
somethingElse()
Whereas node.js focus on providing only asynchronous APIs, Go usually encourages you to write only synchronous APIs (without callbacks or channels). The call to doSomething1 will block the current goroutine and doSomething2 will only be executed after doSomething1 has finished. But that's not a problem in Go, since there are usually other goroutines available that can be scheduled to run on the system thread. In this case, somethingElse is part of another goroutine and can be executed in the meantime, just like in the node.js example.
I personally prefer the Go code, since it's easier to read and reason about. Another advantage of Go is that it also works well with computation heavy tasks. If you start a heavy computation in node.js that doesn't need to wait for network of filesystem calls, this computation basically blocks your event loop. Go's scheduler on the other hand will do its best to dispatch the goroutines on a few number of system threads and the OS might run those threads in parallel if your CPU supports it.
What I feel is Golang can also do callback as same as Node.js but in a synchronous way. As we can pass anonymous function and do closure things
So, why are they comparing Golang and Node.js in callback perspective as if Golang cannot become callback hell.
Yes, of course it is possible to mess things up in Go as well. The reason why you don't see as much callbacks as in node.js is that Go has channels for communication, which allow for a way of structuring your code without using callbacks.
So, since there are channels, callbacks are not used as often therefore it is unlikely to stumble over callback infested code. Of course this doesn't mean that you cannot write scary code with channels as well...
Related
Kind of a weird question, Imagine you have a situation where you need to run 10 SYNCRONOUS functions, it doesn't matter when they complete, you just want to know when all 10 are done: I.E.
f1()
f2()
f3()
...
f10()
doStuffWithResult();
Now, If you use promises like so, assuming you have rewrote each as promoises:
Promise.All([f1,f2,f3,f4,f5,f6,f7,f8,f9,f10])
.then(() => {
doStuffWithResult();
})
Would you see a performance increase? Theoretically, I want to say no because these functions are still synchronous, and everything is still running on one thread.
Thanks!
Would you see a performance increase?
No, what you are proposing would not be faster.
Promises do not create threads. All they do is provide a cooperative system for keeping track of when asynchronous operations are complete and then notifying interested parties of success or failure. They also provide services for propagating errors when asynchronous operations are nested.
And, your proposed Promise.all() code would not even work. You must pass an array of promises to Promise.all(), not an array of function references. In your example, your functions would not even be called.
And, if you changed your code to something that would actually execute, then it would likely be slower than just calling the synchronous functions directly because you'd be executing promise code and all .then() handlers execute on a future tick (not synchronously).
In node.js, the only way to execute synchronous things in parallel is to launch child processes (that can execute in parallel) or pass the operations to some native code that can use actual OS threads.
Does Go have an equivalent of node.js' "emitter"?
I'm teaching myself Go by porting over a node.js library I wrote. In the node version, the library emits an event once something happens (e.g. it listens on UDP port 1234 and when "ABC" is received, "abcreceived" is emitted so the calling code can respond as necessary (e.g. sending back "DEF")
I've seen channels in Go (and am currently reading up on them), but as I'm still new to this language, I don't know if (or how, for that matter) that can be used to communicate with whatever code is using my library.
I've also seen https://github.com/chuckpreslar/emission, but am not sure if this is acceptable, or if there's a better ("Best practice") way of doing things.
Go and Node.js are very different. Node.js supports concurrency only via callbacks. There might be various ways of dressing them up, but they're fundamentally callbacks.
In Node.js, there is no parallelism; Node.js has a single-threaded runtime. When Node.js async is used to achieve what is called 'parallel' execution, it isn't parallel in the sense used in Go, but concurrent.
Concurrency is not parallelism in the Go world.
Go has explicit concurrency based on Communicating Sequential Processes (CSP), a mathematical basis conceived by Tony Hoare at Oxford. The runtime interleaves cooperating processes called goroutines by time-slicing them onto the available CPU cores. Within each goroutine, the code is single threaded, so is easy to write. In the simple case, no data is shared between goroutines; instead messages pass between them along channels. In this way, there is no need for callbacks.
When goroutines get blocked waiting for I/O, that's OK because they don't use any CPU time until they're unblocked. Their memory footprint is slight and you can have very large numbers of them. So callbacks are not needed for I/O operations either.
Because the execution models of Go and Node.js are about as different as they could be, attempting to port code from one to the other is very likely to lead to very clumsy solutions. It's better to start from the original requirements and implement from scratch.
It would be possible to distort the Go concurrency model using function arguments to behave like callbacks. This would be a bad idea because it would not be idiomatic and would lose the benefits that CSP gives.
So by reading others' Go code and some links in the comments to my question, I think channels are the way to go.
In my library code (semi pseudo-code):
// Make a new channel called "Events"
var Events = make(chan
func doSomething() {
// ...
Events <-"abcreceived" // Add "abcreceived" to the Events channel
}
And in the code that will use my library:
evt := <-mylib.Events
switch evt {
case "abcreceived":
sendBackDEF()
break
// ...
}
I still prefer node.js' EventEmitter (because you can transfer data back easily) but for simple things, this should suffice.
I've been doing a fair amount of work with Node lately, trying to build a system which has certain characteristics, one of which is non-blocking / parallelism - a Node strong suit, as I understand it.
What I don't fully understand is when a separate thread is spun off to handle some processing. I'm pretty sue this happens on a function call/call back, but certainly not all of them.
In my specific case, it's an Express based app. At app start-up it does several things including instantiating a RabbitMQ based "bus", an object with a method which will write to the bus (objA) and object which will subscribe to the bus and process messages coming across it (objB).
objA will write to the bus inside an express callback
app.put((req,res) => {
objA.methodWhichWritesToBus();
});
I believe at this point, that objA.methodWhichWritesToBus is executed in a background/worker thread - whatever you call it, not on the main event loop.
Is that the only point at which this sort of thing happens? methodWhichWritesToBus is IO instensive (it calls an elastic search service on another box and brings back 10's to 100's of thousands of records) with lots of chained promises etc., but none of that gets split off, does it?
How about the fact that the obj on which the method is called is instantiated outside the Express callback - does that affect the parallel-ism?
Finally, are the ways to effect/force a method etc to "run in the background"?
I've been noodling this, testing it, for awhile now but all on one machine so it's difficult to tell what's going on.
Who can clarify this for me?
Pre-answer: this is a topic best learned by going and reading, doing coding exercises to solidify your understanding, and working with the technology in a significant way. You're not going to "get it" based on a Q&A format. That said...
What I don't fully understand is when a separate thread is spun off to handle some processing.
Never, sort of. "Processing" as in the computation that happens in your javascript program, happens in the main event loop thread. End of story. However, waiting on I/O to come back from the OS is not considered "processing" so there are various queues managed by node and the OS to track pending I/O requests and invoke callbacks when data is ready. There are a handful of threads node uses internally to manage this stuff with the OS, but from your program's perspective, those threads are irrelevant. Your program can ask node to do some IO, then your program keeps running in parallel, and when the I/O is done, node will eventually invoke the callback in the main event loop and you can process the results.
I believe at this point, that objA.methodWhichWritesToBus is executed in a background/worker thread - whatever you call it, not on the main event loop.
You call it "asynchronously" and it happens whenever you do IO, including filesystem calls, networking, or child processes. Which is to say, quite a lot.
How about the fact that the obj on which the method is called is instantiated outside the Express callback - does that affect the parallel-ism?
Nope.
Finally, are the ways to effect/force a method etc to "run in the background"?
Generally I/O is done asynchronously by default, so no you don't normally need to force anything to run in the background. It's baked into the node design by way of the node core APIs themselves. However, there are ways to delay synchronous processing to a future event loop using setImmediate, setTimeout, or process.nextTick. I explain these in some detail in my blog post setTimeout and friends.
More precisely, all networking is asynchronous. End of story. Specifically, the APIs in node core that are available are all asynchronous, and there's simply no synchronous API available in node. For filesystem IO and child processes, there are both synchronous and asynchronous APIs, but the synchronous APIs must only be used under special limited circumstances, and if you don't know confidently that it's OK in this specific case to make a synchronous IO API call, you should use the asynchronous API so you don't break the lynchpin that makes node perform as it does.
I'm fairly familiar with nodejs now, but I have never tried to build a module before. I was curious to a bit abut async functions.
If you are writing a function that just returns a value, if it worth it to make it async for example, should this be written async?:
exports.getFilename = function () {
return filename;
}
Next, when writing a async function, is writing a function with a callback enough for performance, or is it recommended to thread it using a threading library as well?
Sorry for the somewhat obvious question, I noramlly am the one calling these functions
Callbacks and asynchronousness are two separate things though they are related since in javascript callbacks is the only mechanism to allow you to manage control flow in asynchronous code.
Weather or not non-asynchronous functions should accept callbacks depend on what the function does. One example of a type of function that is not asynchronous but is useful to supply a callback is iteration functions. Array.each() is a good example. Javascript doesn't allow you to pass code blocks so you pass functions to your iteration function.
Another example is filter functions that modify incoming data and return the modified version. Array.sort() is a good example. Passing a function to it allows you to apply your own conditions for how the array should be sorted.
Actually, filtering functions have a stronger reason for accepting functions/callbacks since it alters the behavior of the algorithm. Iteration functions are just nice syntactic sugar around for loops and are therefore a bit redundant. Though, they do make code nicer to read.
Weather or not a function should be asynchronous is a different matter. If it does something that takes a long time to compute (like I/O operations or large matrix calculations) then it should be made asynchronous. How long is "long" depends on your own tolerance. Generally for a moderately busy website a request shouldn't take more than 100ms to complete (in other words, you should be able to handle 10 hits per second at minimum). If an operation takes longer than that then you should split it up and make it async otherwise you'll risk making the site unresponsive to other users. For really busy websites you shouldn't tolerate operations that take longer than 10ms.
From the above explanation it should be obvious that just accepting a function or callback as an argument does not make a function asynchronous. The simplest pure-js way to make something async is to use setTimeout to break long calculations. Of course, the operation still happens in the same thread as the main Node process but at least it doesn't block other requests. To utilize multi-core CPUs on your servers you can use one of the threading libraries on NPM or clusters to make your function async.
Doesn't code take an efficiency hit by being synchronous? Why is coding synchronously a win?
I found these two links in doing some research: http://bjouhier.wordpress.com/2012/03/11/fibers-and-threads-in-node-js-what-for/, https://github.com/Sage/streamlinejs/
If the goal is to prevent spaghetti code, then clearly you can have asynchronous code, with streamline.js for example, that isn't a callback pyramid, right?
You have to distinguish two things here:
Synchronous functions like node's fs.readFileSync, fs.statSync, etc. All these functions have a Sync in their names (*). These functions are truly synchronous and blocking. If you call them, you block the event loop and you kill node's performance. You should only use these functions in your server's initialization script (or in command-line scripts).
Libraries and tools like fibers or streamline.js. These solutions allow you to write your code in sync-style but the code that you write with them will still execute asynchronously. They do not block the event loop.
(*) require is also blocking.
Meteor uses fibers. Its code is written in sync-style but it is non-blocking.
The win is not on the performance side (these solutions have their own overhead so they may be marginally slower but they can also do better than raw callbacks on specific code patterns like caching). The win, and the reason why these solutions have been developed, is on the usability side: they let you write your code in sync-style, even if you are calling asynchronous functions.
Jan 25 2017 edit: I created 3 gists to illustrate non-blocking fibers:
fibers-does-not-block.js, fibers-sleep-sequential.js, fibers-sleep-parallel.js
The code is not "synchronous" when using something like streamlinejs. The actual code will still run asynchronously. It's not very pretty to write lots of anonymous callback functions, thats where these things helps.