I am experimenting with Meteor, and fall in love with it. My question is:
Can I install any nodeJS package, such as NodeMailer, on my Meteor and get it work out of the box? If not, what are usual steps to get it running?
The short answer is no, Meteor has a dependency on Fibers which breaks native compatibility with many pacakges. Currently, getting a package that doesn't use Fibers to work within Meteor is done case-by-case. Here is a recent example
You can follow these instructions to ensure your packages get deployed with your meteor app.
Here is rationale for why Meteor is built with Fibers by David Greenspan, one of Meteor's core devs, emphasis mine:
There's actually no inherent or obvious performance trade-off [between using Fibers or not], so we chose to expose the simpler API (or both).
The Node model is basically your app gets one thread, the event loop. If you want your app to be fast, the request handler had better get off the thread fast! The way it does this in vanilla Node is by finishing and returning, after passing a callback somewhere if there is more work to be done. With fibers, the request handler can instead "yield" when it's doing I/O, so it gets off the thread, but invisibly to the programmer. It's as if there are callbacks happening inside the synchronous calls, but the callback is just the continuation of the program. This is what streamline is trying to simulate, but it happens at the V8 level.
The important point is that Meteor's "synchronous" calls don't block, they yield to the event loop. Whereas normally the event loop would bounce around between whatever callbacks need calling, it instead bounces between whatever functions need further execution.
Related
I've been struggling with this concept:
You can run rust programs in nodejs.
So we all know rust is fast and can handle concurency well and has static memory management and etc...
So maybe I am dumb to understand this concept. Nodejs is a single thread. And everything is running in a event loop.
So how this is working if we have a program that wrote in rust and has multi proccess involing and etc...
What is the behavior of nodejs if we run the program inside the nodejs?
NodeJs' addon API (N-API) is documented here. From the section Asynchronous thread-safe function calls:
JavaScript functions can normally only be called from a native addon's main thread. If an addon creates additional threads, then N-API functions that require a napi_env, napi_value, or napi_ref must not be called from those threads.
When an addon has additional threads and JavaScript functions need to be invoked based on the processing completed by those threads, those threads must communicate with the addon's main thread so that the main thread can invoke the JavaScript function on their behalf. The thread-safe function APIs provide an easy way to do this.
So most calls aren't safe from non-addon-main threads (basically anything affecting the Javascript runtime) though a few calls are dedicated for helping with that.
many question on stackoverflow and others website , some ones says NodeJS is Singlethread and someone says NodeJS is Multithread, and they have there own logic to be Singlethread or Multithread. But If a interviewer ask same question. what should I say. I am getting confusion here.
The main event loop in NodeJs is single-threaded but most of the I/O works run on separate threads.
You can make it multi-threaded by creating child processes.
There is a npm module napajs to create a multi-threaded javascript runtime.
However,the 10.5.0 release has announced multithreading in Node.js. The feature is still experimental and likely to undergo extensive changes, but it does show the direction in which NodeJs is heading.
So stay tuned!!
NodeJS runs JavaScript in a single threaded environment. You get to use a single thread (barring the worker_threads module, and spawning processes).
Under the scenes, NodeJS uses libuv, which uses OS threads for the async I/O you get in the form of the event loop.
Do they run on the same thread as the node process or do they run on a different thread? The CPU usage for my node process goes above 100% when I do a lot of sets and gets (and add several event listeners) on firebase. Any ideas why?
There is only 1 main thread in your Node application, all of your code and all library javascript code will run in this thread. When a library attempts to use IO, it may call upon libuv which runs in its own thread. Libraries may also have their own C++ bindings. Without knowing what you're doing and how the 3rd party library is implemented, it's impossible to tell.
The NodeJS website says the following. Emphasis is mine.
Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.
Even though I love NodeJS I dont see why it is better for scalable applications compared to the existing technologies such as Python, Java or even PHP.
As I understand the JavaScript run-time always runs as a single thread in the CPU. The IO however probably uses underlying kernel methods which might rely on the thread pools provided by the kernel.
So the real questions that need to be answered are:
Because all JS code will run in a single thread NodeJS is unsuitable for applications where there is less IO and lots of computation ?
If I am writing a web application using nodejs and there are 100 open connections each performing a pure computation requiring 100ms, at least one of them will take 10s to finish ?
If your machine has 10 cores but if you are running just one nodeJS instance your other 9 CPUs are sitting ducks ?
I would appreciate if you also post how other technologies perform viz a viz NodeJS in these cases.
I haven't done a ton of node, but I have some opinions on this. Please correct if I am mistaken, SO.
Because all JS code will run in a single thread NodeJS is unsuitable for applications where there is less IO and lots of computation ?
Yeah. Single threaded means if you are crunching lots of data hard in your JS code, you are blocking everything else. And that sucks. But this isn't typical for most web applications.
If I am writing a web application using nodejs and there are 100 open connections each performing a pure computation requiring 100ms, at least one of them will take 10s to finish?
Yep. 10 seconds of CPU time.
If your machine has 10 cores but if you are running just one nodeJS instance your other 9 CPUs are sitting ducks?
That I'm not sure about. The V8 engine might have some optimizations in it that take advantage of multiple cores, transparent to the programmer. But I doubt it.
The thing is, most of the time a web application isn't calculating. If your app is engineered well, a single request can be responded to very quickly. And if you have to fetch things to do that (db, files, remote services) you shouldn't have to wait for that fetch to return before processing the next request.
So you may have many requests in various stages at the same time in various stages of completion, due to when I/O callbacks happen. Even though only one request is running JS code at a time, that code should do what it needs to do very quickly, exit the run loop, and await the next event callback.
If your JS can't run quickly, then this model does pose a problem. As you note, things will get hung as the CPU churns. So don't build a node web application that does lots of intense calculation on the fly.
However, you can refactor things to be asynchronous instead. Maybe you have a standalone node script that can do the calculation for you, with a callback when it's done. Your web application can then boot up that script as a child process, tell it do stuff, and provide a callback to run when it's done. You now have sort of faked threads, in a round about way.
In virtually all web application technologies, you do not want to be doing complex and intense calculation on the fly. Even with proper threading, it's a losing battle. Instead you have to strategize. Do the calculations in the background, or at regular intervals on a cron job, outside of the main web application process itself.
The things you point out are flaws in theory, but in practice it really only becomes an issue if you aren't doing it right.
Node.js is single threaded. This means anything that would block the main thread needs to be done outside the main thread.
In practice this just means using callbacks for heavy computations the same way you use callbacks for I/O.
For instace here's the API for node bcrypt
var bcrypt = require('bcrypt');
bcrypt.genSalt(10, function(err, salt) {
bcrypt.hash("B4c0/\/", salt, function(err, hash) {
// Store hash in your password DB.
});
});
Which Mozilla Persona uses in production. See their code here.
My understanding is that Node is an 'Event' driven as opposed to sequentially driven server application. This I've come to understand means, that for event driven software, the user is in command, he can create an event at any time and the server is in a state such that it can respond, whereas with sequential software (like a DOS prompt), the application tells the user when its 'ok' to response, and may at any given time be not available (due to some other process).
Further, my understanding is that applications like Node and EventMachine use a reactor of sorts.. they wait for an 'event' to occur, and using a callback they delegate the task to some other worker. Ok.. so then, what about Rails & Passenger?
Rails might use a server like NGINX with Passenger to spawn new processes when 'events' are received by the system. Is this not conceptually the same idea? If it is, is it just the processing overhead that is really separating the two where Passenger would need to potentially spawn a new rails instance while, node is already waiting to handle the request?
Node.js is event driven non blocking programming language. The key is the non blocking part. Node doesn't spawn for other processes. It runs in one thread (this is for starters... you can actually spawn it now through some modules - i think - but that's another talk)
Anyway this is different from other typical programming languages where you receive a request and the thread is locked until it has an answer. If you assign it to another thread, that thread is still locked...
In node you never lock. You receive request and the thread continues to receive requests. When a request is processed, the callback is called.
Hope I made myself understand and I used the right terms ;)
Anyway, if you want this video is nicee: http://www.youtube.com/watch?v=jo_B4LTHi3I
The non-blocking/evented I/O as jribeiro described is one part of the answer. Ruby applications tend to be written using blocking I/O, and using processes and threads for concurrency.
However, non-blocking and evented I/O are not inherent to Node. You can achieve the same thing in Ruby by using EventMachine, and an in-process evented server like Thin. If you program directly against EventMachine and Thin, then it is architecturally almost the same as Node. That being said, the Ruby ecosystem does not have as many event-friendly libraries and documentation as Node does, so it takes a bit of skill to achieve the same thing in Ruby.
Conversely, the way Phusion Passenger manages processes - i.e. by spawning multiple processes and load balancing requests between them, and supervising processes - is not unique to Ruby. In fact, Phusion Passenger introduced Node.js support recently. The Node.js support was open sourced today.