I working on a function implementation that is going to be distributed as a npm package.
Once this package has been installed by a consumer as a devDependency, they should be able to load a configuration file by calling a function (let's call it setConfig) from their test files.
I am trying to understand how to resolve a relative path (relative to the test file) when the consumer invokes my function (setConfig('./../some/relative/path/to/config')).
For instance, in the following project structure:
|- node_modules
|- my-published-package
|- dist/file.js (<- this hold setConfig)
|- src
|- configToLoad.json
|- __tests__
|- someTest.spec.js
The config could be specified as ../configToLoad.json and the setConfig should be able to resolve the path relatively to the test file path. In other words, the test author would write `importedClassFromPackage.setConfig('../configToLoad.json').
The setConfig function has the following signature:
setConfig(configPath: string): void {}
Where configPath can either be:
a relative path (from the test file that invokes that function)
an absolute path (absolute path of the configuration file)
a module specifier (i.e my-module/path/to/config.json)
First off, when your setConfig() function runs, you have no idea at all where it was called from. You do not and cannot know the directory of the file that you were called from.
As such, it is the responsibility of the caller to give you a path that you you can just use directly. It needs to either be a fully qualified path or be a relative path that is relative to the current working directory. It cannot be a path that is relative to the directory that the caller's module was in (unless that happens to be the same as the current working directory) because you will have no idea what the caller's module directory is.
So, in your desired spec where you want configPath to be any of these:
a relative path (from the test file that invokes that function)
an absolute path (absolute path of the configuration file)
a module specifier (i.e my-module/path/to/config.json)
Only the absolute path option will work or can be made to work.
The relative path won't work because your function does not know the directory of the test file you were called from.
The module specifier won't work because you don't know how to get the directory of that module name and it's ambiguous because it is possible to have the same named module loaded from more than one place (this happens in the real world when dependencies use a common module, but are using different versions of it).
Note, the caller can use path.join() to join their own module directory with the name of the file and pass you that fully qualified path. In a CommonJS module, the caller gets their own directory from __dirname. In an ESM module, the caller has to extract the directory from import.meta.url.
Related
When I did node myfile.js I got an error that a module was not found. I checked module.paths like this:
$ node
> module.paths
The path where the module is installed showed up and I could require the module:
$ node
> require("the-module")
So I added console.log(module.paths) to myfile.js to see if module.paths was the same. It was not.
What is going on?
EDIT:
It was a bit more complicated.
The module.paths are different, but that does not seem to be the problem. The path where the-module is installed is in the list.
When I add require("the-module") in myfile.js it works as expected. However I do require("c:/path-to/other-file.js") and from there I do another require("the-module"). This require fails.
EDIT 2:
module.paths is other-file.js is different. Why? (And the path where the-module is installed is not in this list.)
You can see all the rules for how require() loads files when you don't specify a full path here.
When you specific only a filename, as in require("the-module"), node.js looks in the node_modules sub-directory below the directory that the current module you are calling require() from was loaded. It is done this way so that a module can have its own set of dependencies and can load them relative to it's own directory without regard from where it is in the file system.
So, if the module you are calling require() from was in c:\myprojects\team\module1 and from within a file in that directory, you call require("the-module"), then node.js will look in c:\myprojects\team\module1\node_modules to find the-module.js.
Similarly, any path that starts with ./ is relative to that same directory, but allows you to reference files in the same directory itself.
node.js does not dynamically add paths to a list of search paths. It sounds like you have an expectation that as you access various directories they are somehow automatically added to a search path. That's now how it works. The rules for where require() looks for relative paths are all spelled out here.
Also, keep in mind that there is not just one module variable across all of node.js. There's a different one in each file that you load in node.js so module.paths very well may be different in each one. In reality, most node.js developers never use module.paths in any way. If you just follow the rules for how relative paths work for require(), you will get what you expect and need.
I am using WebStorm 2017.3 for Node.JS development, and I have a question about source roots.
I use app-module-path to reduce the number of relative paths between my files in require statements, to get rid of stuff like require('../../../utils/common/stuff'). What I do is, on the first line of my app I call
require('app-module-path').addPath(__dirname);
Which makes it possible to do things like require('src/utils/common/stuff'), if the src folder is located directly under the path added to app-module-path.
However, when using these custom source roots, while the code itself works, WebStorm seems unable to resolve the required files. Let's say I have this structure:
app.js
node_modules
|- some_module
|- xyz.js
src
|- routes
|- foo.js
|- bar.js
|- utils
|- stuff.js
WebStorm can of course successfully resolve the following "normal" require statements from foo.js:
require('./bar')
require('../utils/stuff')
require('some_module/xyz')
The first two are of course relative, the last one is because node_modules is a source root (or whatever term is used).
If I now add the root path to app-module-path, all my files can successfully do:
require('src/utils/stuff')
...which I think looks much nicer than endless relative ../../../../ everywhere. However, while the code itself works fine since src/utils/stuff is found relative to the root folder, the file is not resolved by the IDE, meaning my required object/function becomes grey instead of purple/yellow, I cannot Ctrl+Click on any symbols within it or use any other IDE nice stuff, which is very annoying.
I've tried marking directories as "Resource roots", but that doesn't make any difference; files required with absolute paths still cannot be resolved relative to resource roots. Basically what I'm after is the ability to configure additional folders to behave just like node_modules, and in the Directories setting, node_modules is marked as "Excluded", so it surely must be a completely separate setting controlling which folders are used as "require root folders" or "require search paths".
I've also tried making a symlink from <root>/node_modules/src to <root>/src, and while that makes it possible to Ctrl+Click on the actual file path 'src/utils/stuff' in the require statement, it still doesn't resolve the object that was required, and it also causes WebStorm to issue a warning about the module not being listed in package.json (which, of course, is correct).
Is there any way to configure WebStorm with additional "require root folders"? Or is there a better way than app-module-path to get rid of "relative path hell" in require statements while still preserving WebStorm resolving/indexing capabilities?
I see the following...
'./folder/'
'../folder/'
'folder/'
'/folder/'
Can anyone explain the different path types? Also is there a way to just autosearch all folders? Refactoring is a pain!
If the exact filename is not found, then node will attempt to load the required filename with the added extension of .js, .json, and then .node.
.js files are interpreted as JavaScript text files, and .json files are parsed as JSON text files. .node files are interpreted as compiled addon modules loaded with dlopen.
A module prefixed with '/' is an absolute path to the file. For example, require('/home/marco/foo.js') will load the file at /home/marco/foo.js.
A module prefixed with './' is relative to the file calling require(). That is, circle.js must be in the same directory as foo.js for require('./circle') to find it.
Without a leading '/' or './' to indicate a file, the module is either a "core module" or is loaded from a node_modules folder.
If the given path does not exist, require() will throw an Error with its code property set to 'MODULE_NOT_FOUND'.
Extracted from https://nodejs.org/api/modules.html#modules_file_modules
Take a look at the documentation: https://nodejs.org/api/modules.html#modules_loading_from_node_modules_folders
If the module identifier passed to require() is not a native module,
and does not begin with '/', '../', or './', then node starts at the
parent directory of the current module, and adds /node_modules, and
attempts to load the module from that location.
If it is not found there, then it moves to the parent directory, and so on, until the root of the file system is reached.
I am building a tool which is basically a node.js module. This tool can be installed globally (with -g option) I have few static files in the module to generate a report. If the module is invoked locally, I can refer to the static files with relative path ./node_modules/<module>/static/filename. But when the tool is invoked as a command, how do I refer to the static files? And how can I determine whether the tool is invoked as a local module or as a command?
When your tool is installed globally, there are usually two entries on Unix systems:
/usr/local/lib/node_modules/<module>/ -> the directory containing all module source files
/usr/local/bin/<script.js> -> symlink to your module executable script
Because Node is using the real path (i.e. after all symlinks were resolved) to resolve a path relative to current module, you don't need to worry about differences between global and local install.
What you do need to handle is the possibly different current working directory where the node process is running. The solution is to resolve the relative path to your static file against the absolute path where your module resides, not to assume that the node process will run in a particular directory as you do in your example.
There are two ways for that:
Using __dirname (API docs), which contains the directory name (path) of the current source file:
var path = require('path');
// assuming this script is in package root directory
var staticFile = path.resolve(__dirname, 'static', 'filename');
// if the script is in lib/ subdirectory, then you want to call
var staticFile = path.resolve(__dirname, '..', 'static', 'filename');
Using require.resolve(), which returns the exact same filename as would be used by a call to require():
// assuming this script is in package root directory
var staticFile = require.resolve('./static/filename');
// if the script is in lib/ subdirectory, then you want to call
var staticFile = require.resolve('../static/filename');
Use the magic variable __dirname. It refers to the directory containing your script file.
http://nodejs.org/api/globals.html#globals_dirname
instead of requireing code relatively, ie starting with ./ or .., i'd like to define a module "globally". For example, take the following package structure:
/src
/index.js
/a.js
/b.js
/lib
/index.js
...
When in src/a.js or src/b.js, to require lib, I would have to do require('../lib') each time. This gets annoying when you start nesting more as you would have to manually resolve ../lib or ../../lib or ../../../lib.
I want to be able to do require('lib'). Can I do this? Or should I just use globals?
Using a non relative path to require your source files is not how node's require is intended to work! Don't try to work around this restriction by placing arbitrary code file in node_modules directory or workaround by changing the NODE_PATH environment variable.
If you want to use require without a path you should extract the required code as a node module and depend on this node module. This leads to better structured code, less complex modules, encapsulated functionality, better testability and easier code reuse.
You can include package dependencies from http or git so there is no requirement to publish node modules you use in npm. Take a look at npm dependencies for more detail.
use module.exports in the index.js file . and place it inside the node_modules folder
if relative path annoy you and you want to use lib always in your application, you can use global variable like this.
var lib = require('./lib');
global.lib = lib;
you can set lib to global variable in your entry point. after then you can access just lib.
but it's pollute global scope. so you have to use carefully.
placing your module in node_modules dont require you to include a path or relative path
EDIT:
if you place a file named package.json inside the module directory, Node will try to parse that file and look for and use the main attribute as a relative path for the entry point. For instance, if your
./myModuleDir/package.json
file looks something like the following, Node will try to load the file with the path
./myModuleDir/lib/myModule.js
:
{
"name" : "myModule",
"main" : "./lib/myModule.js"
}
If that folder does not contain a package definition file named package.json, the package entry point will assume the default value of index.js, and Node will look, in this case, for a file under the path ./myModuleDir/index.js.