I'm using some external libraries intended to be used in a browser and they set global variables implicitly like a='a' (without the var).
It seems like when I require certain scripts that do this, sometimes the variable will be accessible outside its scope just like in a browser, but for other scripts the global variable is not accessible outside its own script.
Anyone know how nodejs handles implicit global variables, and why I'm seeing somewhat random behavior? I found surprisingly little on the internet.
I can go into the scripts. write something like
if(typeof exports !== 'undefined' && this.exports !== exports){
var GLOBAL=global;
}
else{
var GLOBAL=window;
}
and then change all implicit references to GLOBAL.reference but these scripts are not my own and every time I want to get the latest version of them I would have to do this over again, which is clearly not desirable.
Using module.exports would be cleaner because then I don't have change all the references, but just add a section of the top of every file that exports the globals, but my original question about how node handles implicit globals is still relevant
I am not sure if this answer will help you, since it is hard to diagnose what is going on with your code, but maybe, some of this reasonings can help you diagnose the actual problem in your code.
The behavior in node is actually similar to that of the browser. If you would declare a variable without the var keyword the variable will be accesible through the global object.
//module foo.js
a = 'Obi-wan';
//module bar.js
require('./foo');
console.log(global.a); //yields Obi-wan
console.log(a); //yields Obi-wan
It is not clear why you say this behavior is not consistent in your code, but if you think about it, the use of global variables is precisely subject to this kind of problems since they are global and everyone could overwrite them at any time, causing as a result this unexpected conditions.
There is one aspect in which node is different from the browser though and that could be affecting the behavior that you see.
In the browser, if you do something like this directly in a JavaScript file:
console.log(this==window); //yields true
But if you do the same thing in a Node.js module:
console.log(this==global); //yields false
Basically, in the outer scope of a Node.js module the this reference points to the current module.exports object.
console.log(this==exports); //yield true
So, chances are that if you are putting data in the global scope (window) in the browser through the use of this, you may end up with a module scope in Node.js instead.
Interestingly, the code inside a function in Node.js behaves pretty much as in the browser, in terms of the use of the global scope.
(function what(){
console.log(this==global); //yields true
})();
This does not directly answer your question but it provides a solution since I don't think it is possible.
I love regexp. They are so powerful:
js = js.replace(/^(\t|\s{4})?(var\s)?(\w+)\s=/gm, function () {
if (arguments[1] || arguments[2]) return (arguments[1] || '') + (arguments[2] || '') + arguments[3] + ' =';
return 'exports.' + arguments[3] + ' =';
});*
JSFiddle here
How does it work? I will retrace my work:
/(\w+)\s=/g will take any var, return 'exports.' + arguments[1] + ' ='; will turn them into an export. Not very good.
/(var\s)?(\w+)\s=/g will take any var, but in the callback we examined first group (var\s). Is it undefined? Then we should export it, else nothing should happen. But what about scopes?
/^(\t|\s{4})?(var\s)?(\w+)\s=/gm now we use indent to determine the scope :)
You should be able to run this regex on your file. Be careful, you need it properly indented, and be aware that I might have forgotten some things.
Ah, the problem was that the global variable that was being declared globally in the browser wasn't being declared via a='a', but with var a='a'. In a browser if the var keyword is used not inside a function it will still declare a global variable. It only declares a local variable if the var keyword is inside a function. Node.js doesn't behave this way, and all var declarations are considered local.
Its ashame node.js does this, it makes it less compatible with browser scripts, for no real reason. (other than allowing people not to have to wrap all their scripts in a function).
Related
The question is simple, how do we make es6 modules act like the ImportScript function used on the web browser.
Explanation
The main reason is to soften the blow for developers as they change their code from es5 syntax to es6 so that the transition does not blow up your code the moment you make the change and find out there are a thousand errors due to missing inclusions. It also give's people the option to stay as is indefinitely if you don't want to make the full change at all.
Desired output
ImportScript(A file path/'s); can be applied globally(implicitly) across subsequently required code and vise-verse inside a main file to avoid explicit inclusion in all files.
ES6 Inclusion
This still does not ignore the fact that all your libraries will depend on modules format as well. So it is inevitable that we will still have to include the export statement in every file we need to require. However, this should not limit us to the ability to have a main file that interconnects them all without having to explicitly add includes to every file whenever you need a certain functionality.
DISCLAIMER'S
(Numbered):
(Security) I understand there are many reasons that modules exist and going around them is not advisable for security reasons/load times. However I am not sure about the risk (if any) of even using a method like "eval()" to include such scripts if you are only doing it once at the start of an applications life and to a constant value that does not accept external input. The theory is that if an external entity is able to change the initial state of your program as is launched then your system has already been compromised. So as it is I think the whole argument around Globalization vs modules boils down to the project being done(security/speed needed) and preference/risk.
(Not for everyone) This is a utility I am not implying that everyone uses this
(Already published works) I have searched a lot for this functionality but I am not infallible to err. So If a simple usage of this has already been done that follows this specification(or simpler), I'd love to know how/where I can attain such code. Then I will promptly mark that as the answer or just remove this thread entirely
Example Code
ES5 Way
const fs = require('fs');
let path = require('path');
/* only accepts the scripts with global variables and functions and
does not work with classes unless declared as a var.
*/
function include(f) {
eval.apply(global, [fs.readFileSync(f).toString()])
}
Main file Concept example:
ImportScript("filePath1");loaded first
ImportScript("filePath2");loaded second
ImportScript("filePath3");loaded third
ImportScript("filePath4");loaded fourth
ImportScript("filePath5");loaded fifth
ImportScript("someExternalDependency");sixth
/* where "functionNameFromFile4" is a function defined in
file4 , and "variableFromFile2" is a global dynamic
variable that may change over the lifetime of the
application.
*/
functionNameFromFile4(variableFromFile2);
/* current context has access to previous scripts contexts
and those scripts recognize the current global context as
well in short: All scripts should be able to access
variables and functions from other scripts implicitly
through this , even if they are added after the fact
*/
Typical exported file example (Covers all methods of export via modules):
/*where "varFromFile1" is a dynamic variable created in file1
that may change over the lifetime of the application and "var" is a
variable of type(varFromFile4) being concatenated/added together
with "varFromFile4".
*/
functionNameFromFile4(var){
return var+varFromFile1;
}
//Typical export statement
exportAllHere;
/*
This is just an example and does not cover all usage cases , just
an example of the possible functionality
*/
CONCLUSION
So you still need to export the files as required by the es6 standard , however you only need to import them once in a main file to globalize their functionality across all scripts.
I'm not personally a fan of globalizing all the exports from a module, but here's a little snippet that shows you how one ESM module's exports can be all assigned to the global object:
Suppose you had a simple module called operators.js:
export function add(a, b) {
return a + b;
}
export function subtract(a, b) {
return a - b;
}
You can import that module and then assign all of its exported properties to the global object with this:
import * as m from "./operators.js"
for (const [prop, value] of Object.entries(m)) {
global[prop] = value;
}
// can now access the exports globally
add(1, 2);
FYI, I think the syntax:
include("filePath1")
is going to be tough in ESM modules because dynamic imports in an ESM module using import (which is presumably what you would have to use to implement the include() function you show) are asynchronous (they return a promise), not synchronous like require().
I wonder if a bundler or a transpiler would be an option?
There is experimental work in nodejs related to custom loaders here: https://nodejs.org/api/esm.html#hooks.
If you can handle your include() function returning a promise, here's how you put the above code into that function:
async function include(moduleName) {
const m = await import(moduleName);
for (const [prop, value] of Object.entries(m)) {
global[prop] = value;
}
return m;
}
I have a node toplevel myapp variable that contains some key application state - loggers, db handlers and some other data. The modules downstream in directory hierarchy need access to these data. How can I set up a key/value system in node to do that?
A highly upticked and accepted answer in Express: How to pass app-instance to routes from a different file? suggests using, in a lower level module
//in routes/index.js
var app = require("../app");
But this injects a hard-coded knowledge of the directory structure and file names which should be a bigger no-no jimho. Is there some other method, like something native in JavaScript? Nor do I relish the idea of declaring variables without var.
What is the node way of making a value available to objects created in lower scopes? (I am very much new to node and all-things-node aren't yet obvious to me)
Thanks a lot.
Since using node global (docs here) seems to be the solution that OP used, thought I'd add it as an official answer to collect my valuable points.
I strongly suggest that you namespace your variables, so something like
global.myApp.logger = { info here }
global.myApp.db = {
url: 'mongodb://localhost:27017/test',
connectOptions : {}
}
If you are in app.js and just want to allow access to it
global.myApp = this;
As always, use globals with care...
This is not really related to node but rather general software architecture decisions.
When you have a client and a server module/packages/classes (call them whichever way you like) one way is to define routines on the server module that takes as arguments whichever state data your client keeps on the 'global' scope, completes its tasks and reports back to the client with results.
This way, it is perfectly decoupled and you have a strict control of what data goes where.
Hope this helps :)
One way to do this is in an anonymous function - i.e. instead of returning an object with module.exports, return a function that returns an appropriate value.
So, let's say we want to pass var1 down to our two modules, ./module1.js and ./module2.js. This is how the module code would look:
module.exports = function(var1) {
return {
doSomething: function() { return var1; }
};
}
Then, we can call it like so:
var downstream = require('./module1')('This is var1');
Giving you exactly what you want.
I just created an empty module and installed it under node_modules as appglobals.js
// index.js
module.exports = {};
// package.json too is barebones
{ "name": "appGlobals" }
And then strut it around as without fearing refactoring in future:
var g = require("appglobals");
g.foo = "bar";
I wish it came built in as setter/getter, but the flexibility has to be admired.
(Now I only need to figure out how to package it for production)
Using node-inspector, I'm unable to set breakpoint in the following node.js code. (Content of main.js)
(function() {
require('underscore');
var doSomething = function(callback) {
callback('doSomething Finished');
}
doSomething(function(x) {
console.log(x);
});
}).call(this);
I can easily set a breakpoint on line 2, line 4 or line 8, however no matter how hard I try the debugger won't let me set a break point on line 5 or line 9. To be clear, I'm using the following commands to run node-inspector
node --debug-brk main.js
node-inspector
I also tried to debug in web storm, however the issue persists. If I remove the line require('underscore');, then the problem immediately goes away and I'm able to set break point inside function body again. The problem also goes away if I remove the outermost closure function. It seems that the interaction between require and file level closure is screwing up the node debugging functionality. Has anyone experienced this problem themselves and / or knows any workarounds to be able to break inside function body?
EDIT: My node js version
Tony:~ $ node --version
v0.10.12
Tony:~ $
I ran exactly into the same issue with the same setup.
I've added a breakpoint after the definition of the target-function (that was the only place i could actually add a breakpoint). When the debugger reached that breakpoint and the function was actually defined, i was able to add breakpoints to the actual target-function...
This may not be the answer that you want to hear as it doesn't explain why you can't set any breakpoints, but I would simply remove your require statement from the closure and place it top-level. I would go even further and recommend that you don't use a closure like the one above at all.
The reason is that node uses its own module system, and unlike Javascript in the browser, declaring variables top-level does not pollute the global namespace. This is where require(...) comes in. So, you gain nothing by wrapping your code in an immediately invoked function (unless of course you want your module to be able to run both client side and server side).
I would guess that the reason that you are not able to set any breakpoints is that the V8 runtime is recognizing an unnecessary closure and then optimizing your code for you. The rewritten code may not have the correct source mapping and so breakpoints cannot be set.
So, two suggestions:
require calls are not like regular statements. They are more similar to import statements in Java and are handled specially by the compiler. They should always be top-level in a node file.
No need to wrap your code in an anonymous function when in Node.
I'm refactoring a large javascript codebase to use RequireJS. Unfortunately, many of the files I'm dealing with are not object-oriented, and cannot return an object without significant modification. Is there a more efficient way to give 'dependent' modules access to the functions and variables contained in a module (without returning an object) ?
I have read about using the exports syntax for defining modules, but it is very unclear whether that would be a valid solution for this situation.
In a defined module, the exports object is what gets exported from the module and passed to whatever module requires it.
Consider this:
define(["exports"], function(exports){
exports.myCustomFunction = function(){};
exports.myCustomObject = {};
exports.myCustomVariable = true;
})
This module will place all the disparate functions and/or objects that you want made public onto the exports object.
At this point RequireJS will use that exports object to pass to a module that requires it:
require(["nameOfCustomModule|filename"], function(myCustomModule){
//evaluates to true
console.log(myCustomModule.myCustomVariable);
})
Here's a simple fiddle. Just bring up your console and you will see the value logged there. http://jsfiddle.net/xeucv/
Hope this clears it up a bit!
How can I define a global function in express.js, that without require I can call it
"How" is simple enough:
global.fnName = function(){ return "hi"; }; // Andreas Hultgren's answer
But you don't need the global prefix; the thing about the global object is ...
fnName = function(){ return "hi"; }; // i.e. don't do: var name = function(){ ... };
console.log(fnName()); // this prints "hi"
console.log(global.fnName()); // this also prints "hi" - it was assigned to global.
"Without require" is a separate consideration: if you don't use require there is no guarantee your "globals" will have been declared by the time you need them. It enforces loading order of dependencies, among other things.
"Why am I" and "Is it correct to" are now hidden questions you should consider. It is accepted in javascript that Global Variables ...
... should be reserved for objects that have system-wide relevance and they should be named to avoid ambiguity and minimize the risk of naming collisions - Angus Croll, Namespacing in Javascript
i.e. global truly is Global: it is used by every author of every plugin or library you pull in to your application, not just you. Naming collisions between global variables break your application. This applies equally in node.js.
Global variables are also thought of as a code smell. In the detail sections below here you will see you can quickly get into trouble by using global variables, and they should really be treated as something that pushes you towards dependency injection and/or namespaces and modules.
Node.js and express - global vars and functions
Here's a good rule: if you upload it to a web server, or share it with other people, don't use global variables.
global is permissible in tiny "Saturday afternoon" apps in node.js with express.js, but tend to cause problems later if they get adopted into production. Therefore:
Modules and exports is best practice.
Injection should also be used to reduce coupling between javascript files. But in all cases you will usually need require to ensure they exist by the time you need them:
You should really consider app.locals data, and/or middleware functions, for anything that is view data related.
// call this as a function with an input object to merge
// the new properties with any existing ones in app.locals
app.locals.({
sayHello: function() { return "hi"; }
});
// now you can also use this in a template, like a jade template
=sayHello()
If you are creating global vars/functions for configuration settings purposes the below comments about namespaces still apply, and there are conventions emerging such as config.json files (still using require) for settings that are globally accessed.
Global variables - simple case
It is simple enough to declare a global variable in javascript, and for a function the process is no different. Simply omit the var keyword which would normally force a local scope on the declaration:
// app.js
blah = "boo";
sayHello = function(string toWho) { return "hello " + toWho; }
getVersion = function() { return "0.0.0.1"; }
// routes/main.js
console.log(blah); // logs: "boo"
console.log(global.blah); // logs: "boo"
console.log(sayHello("World")); // logs: "hello World"
console.log(global.sayHello("World")); // logs: "hello World"
console.log(getVersion()); // logs: "0.0.0.1"
But what if two separate plugins in your project use a global getVersion function - how do you get the right version number? Also, how do you ensure that getVersion exists before you need it, or exists at all?
Why do we need require?
To quote the nodejitsu docs the built in require function ...
... is the easiest way to include modules that exist in separate files. The basic functionality of require is that it reads a javascript file, executes the file, and then proceeds to return the exports object
"So", you may ask, "require just makes sure that a module from another file is included? Why bother?" It's better than that: you can make a whole folder a module, making your code easier to organise and test test test, it will recognise various extensions for file modules, not just .js, and it will look in various folders as well. Of course, it caches as well.
So, now that require has found your module, it ensures the code inside it is executed, and puts the objects your created into a "namespace":
// module file ./myModule.js
exports.blah = "boo";
exports.sayHello = function(string toWho) { return "hello " + toWho; }
// routes/main.js
var demoModuleReference = require('./myModule.js');
console.log(demoModuleReference.blah); // logs: "boo"
console.log(demoModuleReference.sayHello("World")); // logs: "hello World"
In that sample, demoModuleReference is an object that looks like:
{
blah: "foo",
sayHello: [Function]
}
Why modules and not global variables (a.k.a namespacing and "Global is the new private")?
Seems complicated now? Surely global variables are easier? requires ensures the following:
It ensures ordered loading of dependencies
It prevents variable name conflicts within global through the exports object.
This application at mankz.com (chrome or firefox only) is fascinating. Depending on how you use your js code, you are very likely to have variable name conflicts within the global scope. Name conflicts come from everywhere. In a browser, for instance, they can come from extensions. node.js is slightly different, but it is becoming more and more extended by compatible plugins as time goes on (you can load jquery in right now, for example). As the versions go on frameworks will get added, and name collisions in global will become more likely. My last run of that app in chrome showed over 1200 global namespace variables.
Namespaces - why?
This global namespace pollution was publicised early on by Douglas Crockford through Eric Miraglia in the article "A JavaScript Module Pattern". In summary:
All objects that need to be used between js files are really global
So, create a namespace object that will be unique
Assign the return value an anonymous function
Add private methods and variables inside that function
Do something useful with the pattern
Example:
ANDYBROWNSONICSUITE.BoomBox.SoundModule = function () {
var privateField = "can't touch this";
return {
play: function() {
console.log(privateField);
}
}
}
Why is this good?
Now you have only increased the global namespace members in the world by one, but this member contains as many items as you like.
Your application is far less likely to clash with other namespaces
It's a pattern, other frameworks expect you to use it to interact with them properly. In that reference, jQuery is a browser plugin, but you can use it with node and therefore your app, so the library interactivity policy statement is a perfect example.
It's a pattern, if we all follow it we our programs are all more likely to get along
When you read the Crockford reference along with the Croll reference (Direct Assignment section) I mentioned at the start, you see why it looks this complicated rather than just doing: sound.play = function() { ... } - ease of maintenance, refactoring the namespace etc. being just one reason.
Summary
In summary:
Can I create globals? Yes, it's simple, omit the var keyword before the declaration.
Should I create globals? You should use the module pattern, which is implicitly supported by node, and by express
Why am I creating globals? If it's for configuration, use a config namespace (e.g. How to store Node.js deployment settings/configuration files?)
You can:
global.name = function(){};
But you really should avoid using globals, even if it's possible to use them.