I have to deal with a node.js server project that uses global variables for common APIs. For instance in the entry point server.js there is a Firebase variable for the real-time database that is stored like this:
fireDB = admin.database();
I wasn't aware that this is possible and I would consider this a bad approach, but now I have to deal with it.
I'm not really interested to re-write any of the many calls to this variable in all those files, rather I would find a way to make fireDB show me suggestions only by changing this variable or installing an extension.
I tried to define it on top of the file as var fireDB, but then suggestions only work in the same file, not in others.
When I set a dot behind admin.database() the suggestions work, when I write fireDB. I get no suggestions, yet the call seems to be possible. Suggestions need to work in other files, too. How can I get this to work?
WARNING: MAKE SURE YOU UNDERSTAND THE PROBLEMS WITH GLOBALS BEFORE USING THEM IN A PROJECT
The above warning/disclaimer is mostly for anyone starting a new project that might happen across this answer.
With that out of the way, create a new .d.ts file and put it somewhere with a descriptive name. For example, globals.d.ts at the top level of the directory. Then just populate it with the following (I don't have any experience with firebase, so I had to make some assumptions about which module you're using, etc.):
globals.d.ts
import { database } from "firebase-admin";
declare global {
var fireDB: database.Database;
}
IntelliSense should then recognize fireDB as a global of the appropriate type in the rest of your JavaScript project.
Why does this work? IntelliSense uses TypeScript even if you're working with a JS project. Many popular JS packages includes a .d.ts file where typings are declared, which allows IntelliSense to suggest something useful when you type require('firebase-admin').database(), for example.
IntelliSense will also automatically create typings internally when you do something "obvious", e.g. with literals:
const MY_OBJ = { a: 1, b: "hello"};
MY_OBJ. // IntelliSense can already autocomplete properties "a" and "b" here
Global autocompletion isn't one of those "obvious" things, however, probably because of all the problems with global variables. I'd also guess it'd be difficult to efficiently know what order your files will run in (and hence when a global might be declared). Thus, you need to explicitly declare your global typings.
If you're interested in further augmenting the capabilities of IntelliSense within your JS project, you can also use comments to explicitly create typings:
/**
* #param {String[]} arrayOfStrings
*/
function asAnExample(arrayOfStrings) {
arrayOfStrings. // IntelliSense recognizes this as an array and will provide suggestions for it
}
See this TypeScript JSDoc reference for more on that.
Related
The question is simple, how do we make es6 modules act like the ImportScript function used on the web browser.
Explanation
The main reason is to soften the blow for developers as they change their code from es5 syntax to es6 so that the transition does not blow up your code the moment you make the change and find out there are a thousand errors due to missing inclusions. It also give's people the option to stay as is indefinitely if you don't want to make the full change at all.
Desired output
ImportScript(A file path/'s); can be applied globally(implicitly) across subsequently required code and vise-verse inside a main file to avoid explicit inclusion in all files.
ES6 Inclusion
This still does not ignore the fact that all your libraries will depend on modules format as well. So it is inevitable that we will still have to include the export statement in every file we need to require. However, this should not limit us to the ability to have a main file that interconnects them all without having to explicitly add includes to every file whenever you need a certain functionality.
DISCLAIMER'S
(Numbered):
(Security) I understand there are many reasons that modules exist and going around them is not advisable for security reasons/load times. However I am not sure about the risk (if any) of even using a method like "eval()" to include such scripts if you are only doing it once at the start of an applications life and to a constant value that does not accept external input. The theory is that if an external entity is able to change the initial state of your program as is launched then your system has already been compromised. So as it is I think the whole argument around Globalization vs modules boils down to the project being done(security/speed needed) and preference/risk.
(Not for everyone) This is a utility I am not implying that everyone uses this
(Already published works) I have searched a lot for this functionality but I am not infallible to err. So If a simple usage of this has already been done that follows this specification(or simpler), I'd love to know how/where I can attain such code. Then I will promptly mark that as the answer or just remove this thread entirely
Example Code
ES5 Way
const fs = require('fs');
let path = require('path');
/* only accepts the scripts with global variables and functions and
does not work with classes unless declared as a var.
*/
function include(f) {
eval.apply(global, [fs.readFileSync(f).toString()])
}
Main file Concept example:
ImportScript("filePath1");loaded first
ImportScript("filePath2");loaded second
ImportScript("filePath3");loaded third
ImportScript("filePath4");loaded fourth
ImportScript("filePath5");loaded fifth
ImportScript("someExternalDependency");sixth
/* where "functionNameFromFile4" is a function defined in
file4 , and "variableFromFile2" is a global dynamic
variable that may change over the lifetime of the
application.
*/
functionNameFromFile4(variableFromFile2);
/* current context has access to previous scripts contexts
and those scripts recognize the current global context as
well in short: All scripts should be able to access
variables and functions from other scripts implicitly
through this , even if they are added after the fact
*/
Typical exported file example (Covers all methods of export via modules):
/*where "varFromFile1" is a dynamic variable created in file1
that may change over the lifetime of the application and "var" is a
variable of type(varFromFile4) being concatenated/added together
with "varFromFile4".
*/
functionNameFromFile4(var){
return var+varFromFile1;
}
//Typical export statement
exportAllHere;
/*
This is just an example and does not cover all usage cases , just
an example of the possible functionality
*/
CONCLUSION
So you still need to export the files as required by the es6 standard , however you only need to import them once in a main file to globalize their functionality across all scripts.
I'm not personally a fan of globalizing all the exports from a module, but here's a little snippet that shows you how one ESM module's exports can be all assigned to the global object:
Suppose you had a simple module called operators.js:
export function add(a, b) {
return a + b;
}
export function subtract(a, b) {
return a - b;
}
You can import that module and then assign all of its exported properties to the global object with this:
import * as m from "./operators.js"
for (const [prop, value] of Object.entries(m)) {
global[prop] = value;
}
// can now access the exports globally
add(1, 2);
FYI, I think the syntax:
include("filePath1")
is going to be tough in ESM modules because dynamic imports in an ESM module using import (which is presumably what you would have to use to implement the include() function you show) are asynchronous (they return a promise), not synchronous like require().
I wonder if a bundler or a transpiler would be an option?
There is experimental work in nodejs related to custom loaders here: https://nodejs.org/api/esm.html#hooks.
If you can handle your include() function returning a promise, here's how you put the above code into that function:
async function include(moduleName) {
const m = await import(moduleName);
for (const [prop, value] of Object.entries(m)) {
global[prop] = value;
}
return m;
}
I'd like to add an important functionality that I can use at work without using require() every time. So, I thought modifying built-in objects can make this happen but I could not locate the locations of those objects to do the modification.
Are those objects in NodeJS binary? Would that mean I have to fork the whole NodeJS repo to make this happen?
Please, I don't want to use prototype editing etc which are same as require. I need to have native feeling.
First off, I agree with an earlier comment that this sounds like a bit of an XY problem where we could better help you if you describe what problem you're really trying to solve. Portable node.js programs that work anywhere or work with any future versions of node.js don't rely on some sort of custom configured environment in order to run. They use the built-in capabilities of node.js and they require/import in external things they want to add to the environment.
Are those objects in NodeJS binary?
Yes, they are in the executable.
Would that mean I have to fork the whole NodeJS repo to make this happen?
Yes.
Please, I don't want to use prototype editing etc which are same as require. I need to have native feeling.
"Native feeling"? This sounds like you haven't really bought into the node.js module architecture. It is different than many other environments. It's easy to get used to over time. IMO, it would really be better to go with the flow and architecture of the platform rather than make some custom version of node.js just to save one line of typing in your startup code.
And, the whole concept of adding a number of globals you can use anywhere pretty much shows that you haven't fully understood the design, architectural, code reuse and testability advantages of the module design baked into node.js. If you had, you wouldn't be trying to write a lot of code that can't be reused in other ways that you don't anticipate now.
That said, in searching through the node.js source code on Github, I found this source file node.js which is where lots of things are added to the node.js global object such as setTimeout(), clearTimeout(), setImmediate(), clearImmediate() and so on. So, that source file seems to be where node.js is setting up the global object. If you wanted to add your own things there, that's one place where it would be done.
To provide a sample of that code (you can see the link above for the complete code):
if (!config.noBrowserGlobals) {
// Override global console from the one provided by the VM
// to the one implemented by Node.js
// https://console.spec.whatwg.org/#console-namespace
exposeNamespace(global, 'console', createGlobalConsole(global.console));
const { URL, URLSearchParams } = require('internal/url');
// https://url.spec.whatwg.org/#url
exposeInterface(global, 'URL', URL);
// https://url.spec.whatwg.org/#urlsearchparams
exposeInterface(global, 'URLSearchParams', URLSearchParams);
const {
TextEncoder, TextDecoder
} = require('internal/encoding');
// https://encoding.spec.whatwg.org/#textencoder
exposeInterface(global, 'TextEncoder', TextEncoder);
// https://encoding.spec.whatwg.org/#textdecoder
exposeInterface(global, 'TextDecoder', TextDecoder);
// https://html.spec.whatwg.org/multipage/webappapis.html#windoworworkerglobalscope
const timers = require('timers');
defineOperation(global, 'clearInterval', timers.clearInterval);
defineOperation(global, 'clearTimeout', timers.clearTimeout);
defineOperation(global, 'setInterval', timers.setInterval);
defineOperation(global, 'setTimeout', timers.setTimeout);
defineOperation(global, 'queueMicrotask', queueMicrotask);
// Non-standard extensions:
defineOperation(global, 'clearImmediate', timers.clearImmediate);
defineOperation(global, 'setImmediate', timers.setImmediate);
}
This code is built into the node.js executable so the only way I know of to directly modify it (without hackish patching of the executable itself) would be to modify the file and then rebuild node.js for your platform into a custom build.
On a little more practical note, you can also use the -r module command line argument to tell node.js to run require(module) before starting your main script. So, you could make a different way of starting node.js from a shell file that always passes the -r fullPathToYourModule argument to node.js so it will always run your startup module that adds things to the global object.
Again, you'd be doing this just to save one line of typing in your startup file. It is really worth doing that?
I'm working with a Node project using VSCode. I'd like to be able to follow the tree upwards from functions/files at a lower level. That is, if I have an exported function doSomething in file dosomething.js, I'd like to see what code calls this function, and/or see what code requires this file.
// index.js
const { doSomething } = require('./tricky-stuff')
// tricky-stuff.js
const doSomething = function() {}
module.exports = {doSomething}
If I'm browsing tricky-stuff.js, is there a built-in feature to VSCode that allows me to see usages of either the function doSomething or where the file tricky-stuff.js is required/imported, so that it would show me the file index.js in this case (as well as perhaps other files where tricky-stuff is used)? Is there any extension? I seem to recall that WebStorm could do this, but I can't remember. (I know I could search for strings, but that seems inelegant).
I think it largely depends on how much VScode knows about your code, via the TypeScript language service or JSDoc comments etc...
Typically you can right click on a function and select the peek all references or find all references options:
In this case, it will show you all the places where that function is called or referenced.
I currently have a project in a loose ES6 module format and my database connection is hard coded. I am wanting to turn this into an npm module and am now facing the issue of how to best allow the end user to configure the code. My first attempt was to rewrite it as classes to be instantiated but it is making the use of the code more convoluted than before so am looking at alternatives. I am exploring my configuration options. It looks like writing to the process env would be the way but I am pondering potential issues, no-nos and other options I have not considered.
Is having the user write config to process env an acceptable method of configuring an npm module? It's a bit like a global write so am dealing with namespace considerations for one. I have also considered using package.json but that's not going to work for things like credentials. Likewise using an rc file is cumbersome. I have not found any docs on the proper methodology if any.
process.env['MY_COOL_MODULE_DB'] = ...
There are basically 5ish options as I see it:
hardcode - not an option
create a configured scope such as classes - what I have now and bleh
use a config such as node-config - not really a user friendly option for npm
store as globals/env. As suggested in comment I can wrap that process in an exported function and thereby ensure that I have a complex non collisive namespace while abstracting that from end user
Ask user to create some .rc file - I would if I was big time like AWS but not in this case.
I mention this npm use case but this really applies to the general challenge of configuring code that is exported as functions. I have use cases for classes but when the only need is creating a configured scope at the expense (in my case) of more complex code I am not sure its worth it.
Update I realize this is a bit of a discussion question but it's helped me wrap my brain around options. I think something like this:
// options.js
let options = {}
export function setOptions(o) { options = o }
export function getOptions(o) { return options }
Then have the user call setOptions() and call this getOptions internally. I realize that since Node requires the module just once that my options object will be kept configured as I pass it around.
NPM modules should IMO be agnostic as to where configuration is stored. That should be left up to the developer, and they may pick their favorite method (env vars, rc files, JSON files, whatever).
The configuration can be passed to your module in various ways. A common way is to export a function that takes an options object:
export default options => {
let db = database.connect(options.database);
...
}
From there, it really depends on what exactly your module provides. If it's just a bunch of loosely coupled functions, you can just return an object:
export default options => {
let db = database.connect(options.database);
return {
getUsers() { return db.getUsers() }
}
}
If you want to allow multiple versions of that object to exist simultaneously, you can use classes:
class MyClass {
constructor(options) {
...
}
...
}
export default options => {
return new MyClass(options)
}
Or export the entire class itself.
If the number of configuration options is limited (say 3 or less), you can also allow them to be passed as separate arguments, instead of passing an object.
Trying to understand what would be the best way to structure some variables. For example in my Node.js Express app, I have the following in app.js:
var poolModule = require('generic-pool');
global.pools = {
/* ... */
};
Where pools is my global variable that keeps track of MySQL and Redis pools. I am also wondering if I can do the same with actual Redis and MySQL objects (and maybe configs variable) so I don't have to require them all over the app. And since they are going to be used the most.
Is this bad practice, and if yes, what's a better way to structure this kind of code?
Edit: added global.
If you require a file you are actually always requiring the same object. So that means you can do:
module.exports = {
// same object for everybody that requires me
};
You have the right idea, but you want to use module.exports to export your object as a module. The CommonJS approach is to have local variables within the module and exported variables for use outside the module. In this way modules can access each others' variables through the use of require. These variables aren't really "global", but in a way are more like "friend" classes in C++. You can in fact have your poolModule do more than store variables for you--you could put methods and other functionality in there too and make it reusable across your whole application.