In my Node app, have a config file like this:
module.exports = {
BUILD_DIR: '/some.path',
OTHER_CONFIG: 42,
// ...
};
I also have some tests doing things like
var appConfig = require('./path/to/appConfig');
appConfig.BUILD_DIR = 'tmp/some/path';
// and stuff done with appConfig
To my big surprise doing this apparently modifies the state of the module. My next test that requires it will have BUILD_DIR set to 'tmp/some/path'. I did not realize modules had this kind of state across requires. How do I get rid of this effect in my test? Also, how can I prevent people from modifying the state of this module? If someone includes it, they should always get what it defines, and not possibly something some other code wrote to it.
The reason why is explained here:
Modules are cached after the first time they are loaded. This means
(among other things) that every call to require('foo') will get
exactly the same object returned, if it would resolve to the same
file.
(emphasis mine)
So the object that you're exporting is cached and shared among the code that uses it.
If that's not what you want, you could export a function instead:
module.exports = function() {
return {
BUILD_DIR: '/some.path',
OTHER_CONFIG: 42,
// ...
};
};
And require it like so:
var appConfig = require('./path/to/appConfig')();
Assuming your module is called 'Config', and you originally:
var Config=require('Config');
you could:
delete require.cache[require.resolve('Config')];
which will remove the module from require's cache, causing it to be loaded afresh when you next 'require' it.
Gist: https://gist.github.com/raisch/6786930
Related
I'm trying to use moment with moment-timer, and it doesn't seem to be able to add the timer extension to the moment I have required.
The top of my module looks like this:
var moment = require('moment');
// this should add timer functions to moment.duration objects...
require('moment-timer');
I can see inside the moment-timer module that the code is set up to require('moment') on its own, so I assume that the moment object that's imported into moment-timer only exists within that module's scope. Because of this, I'm also assuming that my module's require('moment') imports a brand new one, which doesn't get extended. I'd rather not use globals for this (and I'm not sure it would even help). How can I get the extended moment into my module to use duration.timer?
update
I've tried various things using the imports-loader, so far to no avail. I will keep hacking at it and share any findings. What I tried (webpack.config.js):
module.exports: {
// ...
rules: [
{
test: require.resolve('moment-timer'),
use: "imports-loader?define=>false,module=>false,moment"
}
]
// ...
}
I have a node toplevel myapp variable that contains some key application state - loggers, db handlers and some other data. The modules downstream in directory hierarchy need access to these data. How can I set up a key/value system in node to do that?
A highly upticked and accepted answer in Express: How to pass app-instance to routes from a different file? suggests using, in a lower level module
//in routes/index.js
var app = require("../app");
But this injects a hard-coded knowledge of the directory structure and file names which should be a bigger no-no jimho. Is there some other method, like something native in JavaScript? Nor do I relish the idea of declaring variables without var.
What is the node way of making a value available to objects created in lower scopes? (I am very much new to node and all-things-node aren't yet obvious to me)
Thanks a lot.
Since using node global (docs here) seems to be the solution that OP used, thought I'd add it as an official answer to collect my valuable points.
I strongly suggest that you namespace your variables, so something like
global.myApp.logger = { info here }
global.myApp.db = {
url: 'mongodb://localhost:27017/test',
connectOptions : {}
}
If you are in app.js and just want to allow access to it
global.myApp = this;
As always, use globals with care...
This is not really related to node but rather general software architecture decisions.
When you have a client and a server module/packages/classes (call them whichever way you like) one way is to define routines on the server module that takes as arguments whichever state data your client keeps on the 'global' scope, completes its tasks and reports back to the client with results.
This way, it is perfectly decoupled and you have a strict control of what data goes where.
Hope this helps :)
One way to do this is in an anonymous function - i.e. instead of returning an object with module.exports, return a function that returns an appropriate value.
So, let's say we want to pass var1 down to our two modules, ./module1.js and ./module2.js. This is how the module code would look:
module.exports = function(var1) {
return {
doSomething: function() { return var1; }
};
}
Then, we can call it like so:
var downstream = require('./module1')('This is var1');
Giving you exactly what you want.
I just created an empty module and installed it under node_modules as appglobals.js
// index.js
module.exports = {};
// package.json too is barebones
{ "name": "appGlobals" }
And then strut it around as without fearing refactoring in future:
var g = require("appglobals");
g.foo = "bar";
I wish it came built in as setter/getter, but the flexibility has to be admired.
(Now I only need to figure out how to package it for production)
In node I see variables initialized global inside modules are getting mixed up [changes done by one request affects the other] across requests.
For Ex:
a.js
var a;
function printName(req, res) {
//get param `name` from url;
a = name;
res.end('Hi '+a);
}
module.exports.printName = printName;
index.js
//Assume all createServer stuffs are done and following function as a CB to createServer
function requestListener(req, res) {
var a = require('a');
a.printName(req, res);
}
As per my assumption, printName function exported from module 'a' is executed everytime a new request hits node and it will have different scope object everytime.
So, having something global inside a module wouldn't be affecting them across requests.
But I see that isn't the case. Can anyone explain how does node handle module exports of functions in specific [way it handles the scope of the cached module exports object] and how to overcome this shared global variables across requests within a module?
Edit [We do async task per request]:
With rapid requests in our live system. Which basically query redis and responds the request. We see wrong response mapped to wrong request (reply [stored in a global var in the module] of a redis look up wrongly mapped to diff req). And also we have some default values as global vars which can be overridden based on request params. Which also is getting screwed up
The first step to understanding what is happening is understanding what's happening behind the scenes. From a language standpoint, there's nothing special about node modules. The 'magic' comes from how node loads files from disk when you require.
When you call require, node either synchronously reads from disk or returns the module's cached exports object. When reading files, it follows a set of somewhat complex rules to determine exactly which file is read, but once it has a path:
Check if require.cache[moduleName] exists. If it does, return that and STOP.
code = fs.readFileSync(path).
Wrap (concatenate) code with the string (function (exports, require, module, __filename, __dirname) { ... });
eval your wrapped code and invoke the anonymous wrapper function.
var module = { exports: {} };
eval(code)(module.exports, require, module, path, pathMinusFilename);
Save module.exports as require.cache[moduleName].
The next time you require the same module, node simply returns the cached exports object. (This is a very good thing, because the initial loading process is slow and synchronous.)
So now you should be able to see:
Top-level code in a module is only executed once.
Since it is actually executed in an anonymous function:
'Global' variables aren't actually global (unless you explicitly assign to global or don't scope your variables with var)
This is how a module gets a local scope.
In your example, you require module a for each request, but you're actually sharing the same module scope across all requrests because of the module caching mechanism outlined above. Every call to printName shares the same a in its scope chain (even though printName itself gets a new scope on each invocation).
Now in the literal code you have in your question, this doesn't matter: you set a and then use it on the very next line. Control never leaves printName, so the fact that a is shared is irrelevant. My guess is your real code looks more like:
var a;
function printName(req, res) {
//get param `name` from url;
a = name;
getSomethingFromRedis(function(result) {
res.end('Hi '+a);
});
}
module.exports.printName = printName;
Here we have a problem because control does leave printName. The callback eventually fires, but another request changed a in the meantime.
You probably want something more like this:
a.js
module.exports = function A() {
var a;
function printName(req, res) {
//get param `name` from url;
a = name;
res.end('Hi '+a);
}
return {
printName: printName
};
}
index.js
var A = require('a');
function requestListener(req, res) {
var a = A();
a.printName(req, res);
}
This way, you get a fresh and independent scope inside of A for each request.
It really depends when in the process do you assign to name.
if between assigning the name to calling requestListener, there is an async method, then you we'll have "race conditions" (I.E. two threads changing the same object at the same time) even though node.js is single-threaded.
this is because node.js will start processing a new request while the async method is running in the background.
for example look at the following sequence:
request1 starts processing, sets name to 1
request1 calls an async function
node.js frees the process, and handles the next request in queue.
request2 starts processing, sets name to 2
request2 calls an async function
node.js frees the process, the async function for request 1 is done, so it calls the callback for this function.
request1 calls requestListener, however at this point name is already set to 2 and not 1.
dealing with Async function in Node.js is very similar to multi-threaded programming, you must take care to encapsulate your data. in general you should try to avoid using Global object, and if you do use them, they should be either: immutable or self-contained.
Global objects shouldn't be used to pass state between functions (which is what you are doing).
The solution to your problems should be to put the name global inside an object, the suggested places are inside the request object, which is passed to all most all functions in the request processing pipelie (this is what connect.js,express.js and all the middleware are doing), or within a session (see connect.js session middleware), which would allow you to persist data between different requests from the same user.
Modules were designed for run once and cache the module, that, combined with node's asynchronous nature means about 50% of the time res.end('Hi '+a) executes before a = name (because a is known).
Ultimately it boils down to one simple fact of JavaScript: global vars are evil. I would not use a global unless it never gets overridden by requests.
I ran into an issue with my Nodejs application.
I have two different apps that are using shared library, which is located so that it is found one level up in node_modules. So i have this structure ./app1/app.js, ./app2/app.js and ./node_modules/shared.libs/index.js.
shared.libs in its turn has some other modules installed, like mongoose, redis etc. Plus some mogoose models with additional functions in them. All are exported from index.js like this:
exports.async = require('async');
exports.config = require('./config');
exports.utils = require('./lib/utils');
And then in apps i import them like this:
var libs = require('shared.libs');
var config = libs.config;
So after this code i can use config which is coming from that shared library.
This part was and is working just fine. But now i need to put additional layer on top of this library (read: provide more unified interface for both apps).
What i tried to do is to add some functions into index.js of shared library and then export the whole object with these functions. But whenever i try to call previously imported (by var libs = require('shared.libs');) object it says that libs is not defined.
What am i doing wrong here?
I generally want to keep other code the same, so i won't need to go over replacing require part everywhere, but at the same time provide additional functionality from shared library which will be available from that imported libs object.
this should definitely work:
module.exports = {
async: require('async'),
config: require('./config'),
utils: require('./lib/utils'),
foo: function () {
return 'bar';
}
};
reference like:
var libs = require('shared.libs');
console.log(libs.async);
console.log(libs.config);
console.log(libs.utils);
console.log(libs.foo);
console.log(libs.foo());
What makes me wonder is one of your comments above, that you get an error libs is not defined. That looks like you should have other unnoticed errors before.. during the the initialization of your shared.libs module..
I have a question related to the node.js documentation on module caching:
Modules are cached after the first time they are loaded. This means
(among other things) that every call to require('foo') will get
exactly the same object returned, if it would resolve to the same
file.
Multiple calls to require('foo') may not cause the module code to be
executed multiple times. This is an important feature. With it,
"partially done" objects can be returned, thus allowing transitive
dependencies to be loaded even when they would cause cycles.
What is meant with may?
I want to know if require will always return the same object. So in case I require a module A in app.js and change the exports object within app.js (the one that require returns) and after that require a module B in app.js that itself requires module A, will I always get the modified version of that object, or a new one?
// app.js
var a = require('./a');
a.b = 2;
console.log(a.b); //2
var b = require('./b');
console.log(b.b); //2
// a.js
exports.a = 1;
// b.js
module.exports = require('./a');
If both app.js and b.js reside in the same project (and in the same directory) then both of them will receive the same instance of A. From the node.js documentation:
... every call to require('foo') will get exactly the same object returned, if it would resolve to the same file.
The situation is different when a.js, b.js and app.js are in different npm modules. For example:
[APP] --> [A], [B]
[B] --> [A]
In that case the require('a') in app.js would resolve to a different copy of a.js than require('a') in b.js and therefore return a different instance of A. There is a blog post describing this behavior in more detail.
node.js has some kind of caching implemented which blocks node from reading files 1000s of times while executing some huge server-projects.
This cache is listed in the require.cache object. I have to note that this object is read/writeable which gives the ability to delete files from the cache without killing the process.
http://nodejs.org/docs/latest/api/globals.html#require.cache
Ouh, forgot to answer the question. Modifying the exported object does not affect the next module-loading. This would cause much trouble... Require always return a new instance of the object, no reference. Editing the file and deleting the cache does change the exported object
After doing some tests, node.js does cache the module.exports. Modifying require.cache[{module}].exports ends up in a new, modified returned object.
Since the question was posted, the document has been updated to make it clear why "may" was originally used. It now answers the question itself by making things explicit (my emphasis to show what's changed):
Modules are cached after the first time they are loaded. This means
(among other things) that every call to require('foo') will get
exactly the same object returned, if it would resolve to the same
file.
Provided require.cache is not modified, multiple calls to
require('foo') will not cause the module code to be executed multiple
times. This is an important feature. With it, "partially done" objects
can be returned, thus allowing transitive dependencies to be loaded
even when they would cause cycles.
For what I have seen, if the module name resolve to a file previosuly loaded, the cached module will be returned, otherwise the new file will be loaded separately.
That is, caching is based on the actual file name that gets resolved. This is because, in general, there can be different versions of the same package that are installed at different levels of the file hierarchy and that must be loaded accordingly.
What I am not sure about is wether there are cases of cache invalidation not under the programmer's control or awareness, that might make it possible to accidentaly reload the very same package file multiple times.
In case the reason why you want require(x) to return a fresh object every time is just because you modify that object directly - which is a case I ran into - just clone it, and modify and use only the clone, like this:
var a = require('./a');
a = JSON.parse(JSON.stringify(a));
try drex: https://github.com/yuryb/drex
drex is watching a module for updates and cleanly re-requires the
module after the update. New code is being require()d as if the new
code is a totally different module, so require.cache is not a problem.
When you require an object, you are requiring its reference address, and by requiring the object twice, you will get the same address! To have copies of the same object, You should copy (clone) it.
var obj = require('./obj');
a = JSON.parse(JSON.stringify(obj));
b = JSON.parse(JSON.stringify(obj));
c = JSON.parse(JSON.stringify(obj));
Cloning is done in multiple ways, you can see this, for further information.