node module.paths confusion
Issue: required node modules do not use my global module.paths even though I add the global path to the array in my main.js application.
Example:
mymain.js
//global modules path
module.paths.push('C:\Users\xuser\AppData\Roaming\npm');
// finds ws in global modules path. Works!
wsmain=require('ws')
// Now load a 3rd party module, which also requires('ws')
C = require('cmod.js');
cmod.js
ws=require('ws'); // fails to find global path
q: How do I make sure that module global is also passed on to the require modules. Is there a way to pass it as a parameter or something?
I'm not sure there is a "global" path. The node.js documentation suggests a default require takes place relative to the file requiring it, and searches for a node_modules directory up the directory chain. From the docs:
For example, if the file at '/home/ry/projects/foo.js' called
require('bar.js'), then node would look in the following locations, in
this order:
/home/ry/projects/node_modules/bar.js
/home/ry/node_modules/bar.js
/home/node_modules/bar.js
/node_modules/bar.js
It looks like you can use NODE_PATH env var to give a list of places to look for modules:
https://nodejs.org/api/modules.html#modules_loading_from_the_global_folders
Based on the documentation, the following might work:
NODE_PATH=/path/to/node_modules/where/ws/lives node mymain.js
Related
For a project with a structure like:
index.js => require('debug')
=> require('express')
node_modules
debug => version 3.x.x
express => require('debug')
node_modules
debug => version 2.x.x
NodeJS loads the debug module required in index.js, with a 3.x.x version
When the debug module is loaded for the express module, does NodeJS uses the already cached debug module (v3.x.x) or it loads the one required by the express module (2.x.x) ?
The module cache is not just by the root filename, it's by their fully resolved path.
So, if module "A" is loading v3.xx of index.js and module "B" is loading v2.xx of index.js from a different disk location, then both will get loaded since the module cache recognizes them as different files and will cache them separately.
So, since v3.xx and v2.xx of index.js can't ever possibly exist at the same full path, the whole system works just fine as each module can require() in the precise version of each dependent module that it comes with.
This assumes that modules are well-behaved and aren't doing things that might ruin this ability such as establishing globals that might conflict or competing for locking of shared resources, etc... If they are well behaved modules, then a project can have one module using v3.xx and another module using v2.xx. This is made possible by the separate scopes of each module and the ability for each dependent module to import them each separately.
I read that in the nodejs documentation and it's not clear for me what do they mean by "same file". Is it same file name , same file absolute path, same file content ?
It's is the same full path so something like:
/whatever/user/someUser/myProject/node_modules/index.js
would be a full path to index.js.
If you don't specify a full path to require() as in require('index.js'); or you specific a relative path, then it will be the full path on which require() finds the index.js according to the search rules specified for require() (as specified in the doc).
Background:
Assuming my folder structure is something like:
project/
-- package.json
-- index.mjs
-- lib/
-- config/
-- index.mjs
When I used require() natively in node I could reference a local module using a bare import like:
const x = require('config')
because I added the root of my library folder to the NODE_PATH environment variable. (assuming of course when I was using cjs/require the extension would have been .js)
The problem:
When I try to do that using native es6 modules (mjs) like:
import x from 'config'
I get the error:
Error [ERR_MODULE_RESOLUTION_LEGACY]: config not found by import in [SOME_ABS_PATH]/index.mjs. Legacy behavior in require() would have found it at [SOME_ABS_PATH]/lib/config/index.mjs
The question:
Anyone know how to solve this? Or what the future is for dealing with relative paths for local module resolution within the node native es6 module system?
Research so far:
Only resource I've found on this so far is from here http://2ality.com/2017/09/native-esm-node.html stating:
Path resolution works slightly differently: ESM does not support NODE_PATH and require.extensions. And its specifiers always being URLs also leads to a few minor differences.
And on the error message below ERR_MODULE_RESOLUTION_LEGACY - google showed up literally nothing.
Okay, so a colleague (thanks #robin-balmforth ) gave me the answer:
From https://nodejs.org/api/esm.html#esm_no_node_path it says:
Notable differences between import and require No NODE_PATH NODE_PATH
is not part of resolving import specifiers. Please use symlinks if
this behavior is desired.
And from something even more canonical:
https://github.com/bmeck/node-eps/blob/es6-module/002-es6-modules.md#521-removal-of-non-local-dependencies says:
All of the following will not be supported by the import statement:
$NODE_PATH $HOME/.node_modules $HOME/.node_libraries $PREFIX/lib/node
Use local dependencies, and symbolic links as needed.
So instead of me setting the NODE_PATH environment variable I have to setup a symlink like:
ln -s ../lib node_modules/lib
Seems to work fine yay.
We suppose the reason is for compatibility with the browser implementation of es6 modules?
There is some discussion on the reasoning behind this change from https://github.com/bmeck in this node-eps issue https://github.com/nodejs/node-eps/issues/11 like:
The solution is not a workaround, it is how path resolution works.
Having a single flat namespace for "bare" path resolution is
important.
I am looking at the following tutorial https://github.com/jakblak/thinkster_mean_app/blob/master/app.js and can see that require('path') is present in the app.js. However, package.json does not seem to explicitly install it? So where is path being taken from? Is it present by default?
path is a NodeJS built-in module, so there is no need to add it over npm. Its already available when you install NodeJS like Console module or os module.
Here is the available path documentation which can give you more details about path itself.
And here is the list of all built-in modules and high-level concepts.
So, I'm not using Node or WebPack serverside, but would still like to use modules from npm every now and then. My clientside uses requirejs, so I would need the modules in either AMD (preferred) or CommonJS.
What I want to archieve is a script that takes the module name + "external dependencies" as arguments and creates a module that contains all the other deps.
E.g.
sh npmtoamd.sh draft-js react react-dom
It creates an ES5 AMD module that contains draft-js and all of it's dependencies excluding react and react-dom. If it's not possible to eg. include css files and other non-js content in the module, providing them in eg. draft-js.css is tolerable.
While I don't use Node or Webpack serverside, we can use npm and webpack in the said script.
Fetching the module from npm is the trivial part, but I'm pretty lost in how to do the webpack parts. I know for a fact that it's possible though, as I managed to do it earlier with help, just don't have it down anywhere and no idea how it went.
I think as elmigranto commented, Browserify is what you're looking for. Unlike its name implies, it can be used in both the browser environment and the node environment. In a nutshell, it does this:
Browserify starts at the entry point files that you give it and searches for any require() calls it finds using static analysis of the source code's abstract syntax tree.
For every require() call with a string in it, browserify resolves those module strings to file paths and then searches those file paths for require() calls recursively until the entire dependency graph is visited.
Each file is concatenated into a single javascript file with a minimal require() definition that maps the statically-resolved names to internal IDs.
This means that the bundle you generate is completely self-contained and has everything your application needs to work with a pretty negligible overhead.
If you check out some of the demos you can see that all the dependencies (and their co-dependencies) are bundled into one file.
A simple example:
browserify main.js -o bundle.js
In regards to using AMD as well Browserify supports it by using deamdify.
Using AMD modules:
browserify -t deamdify main.js -o bundle.js
I ended up doing the npm fetch thingy in java instead of a batch script and finally got it working. Didn't get browserify to work however.
Heres what I do:
creating the following webpack.config.js
module.exports = {
entry: './main.js',
output: {
filename: 'bundle.js',
library:"<modulename>",
libraryTarget:"amd"
},
externals: {
react: "React",
"react-dom": "ReactDOM",
immutable:"Immutable",
moment:"Moment"
}
};
npm install <modulename>
Creating the following main.js
define('FOOBAR', ['<modulename>'], function(Foo) {
return Foo;
});
Running webpack main.js
I'm migrating a Node + Express application to TypeScript and am trying to use external modules. I was using the symlink trick to not have to worry about relative paths.
Works (no tsc warning):
var authz = require('LIB/app/authorization');
import authz = require('../..//app/authorization');
Doesn't work with symlink (tsc gives warning TS2307: Cannot find module 'LIB/app/authorization'.):
var authz = require('LIB/app/authorization');
I believe it doesn't work due to this TypeScript issue. My question is, which of the possible options to avoid relative path require statements works in TypeScript. Is there a preferred option?
Do what issacs says : https://gist.github.com/branneman/8048520#gistcomment-972519
Have different modules +win win+. Note : you need typescript nightly to be able to use .ts files from different node_modules.
Alternative
grunt-ts transforms https://github.com/TypeStrong/grunt-ts#transforms take the burden off of relative paths
Otherwise don't use hacks and use relative paths. You IDE should provide completions for you (checkout https://github.com/TypeStrong/atom-typescript