If, as part of a NodeJS file, there are different closures:
const Library2 = require('Library2'); // should it be here?
doSomething().then(()=>{
const Library1 = require('Library1'); // or here?
return Library1.doSomething();
}).then(()=>{
return Library2.doSomething();
}).then(...) // etc.
Would it be better to require Library1 and Library2 in the scopes in which they are used? Or at the top of the file like most do?
Does it make a difference to how much memory is consumed either way?
It is best to load all modules needed at server startup time.
When a module is loaded for the first time, it is loaded with blocking, synchronous I/O. It is bad to ever use blocking, synchronous I/O during the run-time for your server because that interferes with the ability of your server to handle multiple requests at once and reduces scalability.
Modules loaded with require() are cached so fortunately, trying to require() in a module in the middle of a request handler really only hurts performance the very first time the request is run.
But, it's still best to load any modules in your startup code and NOT during the run-time request-handling of your server.
Related
We are integrating Amazon's node.js SDK into our project and while I do not think it matters due to require's cache and the fact that everything is compiled, I could not find a site that definitively states that multiple requires will not affect performance in run time.
Obviously it depends on what files you are requiring, the contents of those files, and whether or not they could block the event loop or have other code inside of them to slow performance.
I prefer to structure code based on functionality rather than just having a 10000+ line file that does not really relate to the task at hand. I just want to make sure I'm not shooting myself in the foot by break out functionality into separate modules and then requiring on an as needed basis.
Well, require() is a synchronous operation so it should ONLY be used during server initialization, never during an actual request. Therefore, the performance of require() should only affect your server startup time, not your request handling time.
Second, require() does have a cache behind it. It matches the fully resolved path of the module you are attempting to load. So, if you call require(somePath) and a module at that same path has previously been loaded, then the module handle is just immediately returned from the cache. No module is loaded from disk a second time. The module code is not executed a second time.
Obviously it depends on what files you are requiring, the contents of those files, and whether or not they could block the event loop or have other code inside of them to slow performance.
If you are requiring a module for the first time, it WILL block the event loop while loading that module because require() uses blocking, synchronous I/O when the module is not yet cached. That's why you should be doing this at server initialization time, not during a request handler.
I prefer to structure code based on functionality rather than just having a 10000+ line file that does not really relate to the task at hand. I just want to make sure I'm not shooting myself in the foot by break out functionality into separate modules and then requiring on an as needed basis.
Breaking code into logical modules is good for ease of maintenance, ease of testing and ease of reuse, so it's definitely a good thing.
I have seen people go too far where there are so many modules each with only a few lines of code in them that it backfires and makes the project unwieldly to work on, find things in, design test suites for, etc... So, there is a balance.
What is the best way to use NodeJS's require function? By this, I'm referring to the placement of the require statement. Isf it better to load all dependencies from the beginning of the script, as you need them, or does it not make a notable difference whatsoever?
This article has a lot useful information regarding how require works, though I still can't come to a definitive conclusion as to which method would be most efficient.
Assuming you're using node.js for some sort of server environment, several things are generally true about that server environment:
You want fast response time to any given request.
The code that runs for processing requests should not use synchronous I/O operations because that seriously lessens the scalability of the server.
Server startup time is generally not something you need to optimize for (within reason) so if you're going to pay an initialization cost somewhere, it is usually better paid once at server startup time.
So, given that require() uses synchronous I/O when the module has not yet been cached, that means you really don't generally want to be doing require() operations inside a request handler. And, you want fast response times for your request handlers so you don't want require() calls inside your handler anyway.
All of these leads to a general rule of thumb that you load necessary modules at startup time into a module level variable that you can reuse from one request to the next and you don't load modules inside your request handlers.
In addition to all of this, if you put all your require() statements in a block near the top of your module, it makes your module a lot more self-documenting about what other modules it depends on and how it initializes those modules. If require() statements are sprinkled all over the code, then it makes it a lot harder for a developer to see what this module is using without a lot more study of the code.
It depends what performance characteristics you're looking for.
require() is not cheap; it has to read the JS file from disk, parse it, and execute any top-level code (and do all of that recursively for all files require()d by that file).
If you put all of your require()s on top, your code may take more time to start, but it won't suddenly slow down later. (note that moving the require() further down in the synchronous top-level code will make no difference beyond order of execution).
If you only require() other modules when first used asynchronously, your first request may be noticeably slower, as Node parses all of your dependencies. This also means that any errors from dependencies won't be caught until later. (note that all require() calls are cached, so the second request won't do any more work)
The other disadvantage to scattering require() calls throughout your code is that it makes it less readable; it's very nice to easily see exactly what each file depends on up top.
I'm working on a project which needs to load specific module based on url, and there are a lot of modules, I'm wondering is it going to blow up the memory if you keep requiring modules? If it is, is there anyway to free up memory after each module is executed? Or is there anyway better than requiring modules? like fs.readFile then eval it?
You can load as many modules as you want, but the hardware limitations will still be there. If your memory is going to blow up - will depend on the memory consumption of each module, the tasks your program is doing, and the amount of RAM you have. If you keep on requiring modules just to test if it breaks, I am sure it will break after a certain number of modules are loaded into the memory.
When you use memory intensive modules, you can scope them i.e. require() them inside a function like: to make sure they are unloaded from memory after the operation is complete.
// this code does image manipulation
function resizeImage(filename, cb) {
require('gm').open(filename).size(function (err, size) {
console.log(size);
// do something else with the image
cb(null);
// gm is queued for garbage collection
// after this operation
});
}
fs.readFile and eval will not be better than require().
If you have all memory optimizations in place and still your RAM is not enough, the application will break, and you will need to scale up your machine.
Recently im working on new Nodejs project and find some codes like this :
function a(){
var http = require('http');
var fs = require('fs');
}
function b(){
var path = require('path');
var http = require('http');
}
function c(){
var fs = require('fs');
}
so i have a some question about coding like this :
is require have some rules for using it?
is it good using rquire on top of codes or when we need it call it.
is coding like this make conflict
Some rules for when to call require:
By default, require a module globally once at the start of the file and do not reassign the variable to which the result of require is assigned.
If requiring a module has been proved to impact performance significantly (maybe it has initialization routines that take a long time to run) and it is not used throughout the file, then require it locally, inside the function that needs it.
If the module's name must be computed in a function, then load it locally.
If the code you show in your question is all in one file and is meant to be used in production, I'd ask the coder who produced it what warrants using require in that way, and if a good, substantial reason, supported by evidence cannot be formulated, I'd tell this coder to move the require calls to the start of the file.
You definitely should avoid these in production. Modules are cached indeed, so it can affect performance only during initial calls to require, but still it can. Also fs, http and and path are built-in modules, so require-ing them doesn't involve reading from disk, only code compilation and execution, but if you use non built-in modules you will also block event loop for the time of reading from disk.
In general, if you use any sync functions, including require, you should use them only during first tick, since no servers are listening yet anyway.
Manning publication has a good book about node, called Node.js in action this is how the node's module requiring rules described it.
What is currently the best practice for loading models (and goes for all required files I guess)?
I'm thinking these two ways to achieve the solution (nonsense code to illustrate follows):
var Post = require('../models/post');
function findById(id) {
return new Post(id);
}
function party() {
return Post.getParty();
}
vs
function findById(id) {
return new require('../models/post')(id);
}
function party() {
return require('../models/post').getParty();
}
Is one of these snippets preferred? Are there considerable memory and time tradeoffs? Or is it just a premature optimization?
It's a premature optimization (calls to require() are cached and idempotent), but I'd personally call your first style better (loading dependencies during initialization rather than subsequent processing) since it's easier to get your head around what you're doing. Loading everything at the start will slightly slow down your startup (which is hardly ever an issue) in return for making most requests run slightly faster (which you shouldn't worry about unless you've identified a bottleneck and done some hardcore profiling).
You should definititely use the version with the single call to requireat the beginning. Although it does not make any difference regarding to how often the modules are loaded (they are only loaded once however you do it), there are performance issues in the second way of doing it.
The problem is that require is one of only a few functions in Node.js that is blocking. That means, as long as it runs, Node.js is not able to fulfill any incoming requests. On startup this is no problem: It only takes a while until your application is up and running.
But for sure you don't want to have blocking moments while your application is already running for a while.
So, if you do not have VERY special reasons for the second option, go with the first one.
I believe the second case is useful to avoid circular module dependencies, since the require() happens at run-time rather than load time.
Otherwise, I believe the first is (slightly?) faster and to me, quite a bit more readable.