I need to calculate a pathname relative to the file-system location of the module. I am using the latest TypeScript on Node.js 12.x. For other reasons in tsconfig.json I have set
"target": "esnext",
"module": "esnext",
This triggers a mode that is strict to the Node.js support for ES6 Modules. In that mode the __dirname variable is not available because that global is not defined in the ESanything spec. What we're supposed to do instead is access the import.meta.url variable and extract the directory name.
For an example see the last answer on this question: Alternative for __dirname in node when using the --experimental-modules flag
But in the TypeScript DefinitelyTyped collection the ImportMeta class is not defined to include a url member. Therefore the code fails to compile:
tdn.ts:7:25 - error TS2339: Property 'url' does not exist on type 'ImportMeta'.
7 console.log(import.meta.url);
~~~
I wasn't able to find the definition of ImportMeta in the Definitely Typed repository. But it seems to be improperly defined.
UPDATE: In node_modules//typescript/lib/lib.es5.d.ts I found this:
/**
* The type of `import.meta`.
*
* If you need to declare that a given property exists on `import.meta`,
* this type may be augmented via interface merging.
*/
interface ImportMeta {
}
Ugh...
/UPDATE
In the Node.js 12.x documentation on the ES Modules page it clearly describes the shape of import.meta and that we're supposed to do something like:
import path from 'path';
import { fileURLToPath } from 'url';
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
'__dirname', '__filename' and 'require'...etc are NodeJS specific keywords, and by default typescript doesn't recognize them, although you need to know that the compiler compile the ts file to js file (by default) and it works fine, in order to clear the errors you can to run this on your terminal (or cmd on windows):
npm install --save #types/node
that would install nodejs types definitions and allow you to avoid this type of errors when writing your nodejs programs in typescript.
using import.meta and fileToURLupdate this works...
As per NODE.org, both CommonJs variables specially __dirname or __filename are not available in ES Modules.
We have to replicate those commonJs variables from via import.meta.url.
Source: https://nodejs.org/api/esm.html#esm_no_filename_or_dirname
Related
I'm using this node module 'deep-equal' () and so far the only way I've been able to access the function in it is with var equal = require('deep-equal');, as suggested in the documentation. I usually use import instead of require, and am wondering if it's possible to use import with this module. The main function is exported from the file in this line:
var deepEqual = module.exports = function (actual, expected, opts) {...}
Is it possible to import this function with import, or is it only possible with require?
Thanks!
Yes you actually can.
If you're using nodejs LTS then you'll have to use .mjs extension for the file where you're using import and pass experimental-modules flag while running node process.
// foo.mjs
import equal from 'deep-equal';
node --experimental-modules foo.mjs
As of nodejs 12.3.0 you can simply pass the experimental-modules. From docs
Once enabled, Node.js will treat the following as ES modules when passed to node as the initial input, or when referenced by import statements within ES module code
Also you can specify type as module in your package.json:
// package.json
{
"type": "module"
}
From docs
Files ending with .js or .mjs, or lacking any extension, will be loaded as ES modules when the nearest parent package.json file contains a top-level field "type" with a value of "module"
I require a module that was installed via npm. I want to access a .js file subordinate to that module (so I can subclass a Constructor method in it). I can't (well, don't want to) modify the module's code, so don't have a place to extract its __dirname.
I am aware of the following question, but it is about getting the path of a module that one has code control over (hence, __dirname is the solution):
In Node.js how can I tell the path of `this` module?
~~~
Even better would be to get the module's loaded module info
If I correctly understand your question, you should use require.resolve():
Use the internal require() machinery to look up the location of a module, but rather than loading the module, just return the resolved filename.
Example: var pathToModule = require.resolve('module');
require.resolve() is a partial answer. The accepted answer may work for many node modules, but won't work for all of them.
require.resolve("moduleName") doesn't give you the directory where the module is installed; it gives you the location of the file defined in the main attribute in the module's package.json.
That might be moduleName/index.js or it could be moduleName/lib/moduleName.js. In the latter case, path.dirname(require.resolve("moduleName")) will return a directory you may not want or expect: node_modules/moduleName/lib
The correct way to get the complete path to a specific module is by resolving the filename:
let readmePath = require.resolve("moduleName/README.md");
If you just want the directory for the module (maybe you're going to make a lot of path.join() calls), then resolve the package.json — which must always be in the root of the project — and pass to path.dirname():
let packagePath = path.dirname(require.resolve("moduleName/package.json"));
FYI, require.resolve returns the module identifier according to CommonJS. In node.js this is the filename. In webpack this is a number.
In webpack situation, here is my solution to find out the module path:
const pathToModule = require.resolve('module/to/require');
console.log('pathToModule is', pathToModule); // a number, eg. 8
console.log('__webpack_modules__[pathToModule] is', __webpack_modules__[pathToModule]);
Then from __webpack_modules__[pathToModule] I got information like this:
(function(module, exports, __webpack_require__) {
eval("module.exports = (__webpack_require__(6))(85);\n\n//////////////////\n//
WEBPACK FOOTER\n// delegated ./node_modules/echarts/lib/echarts.js from dll-reference vendor_da75d351571a5de37e2e\n// module id = 8\n// module chunks = 0\n\n//# sourceURL=webpack:///delegated_./node_modules/echarts/lib/echarts.js_from_dll-reference_vendor_da75d351571a5de37e2e?");
/***/
})
Turned out I required old scripts from previous dll build file(for faster build speed), so that my updated module file didn't work as I expected. Finally I rebuilt my dll file and solved my problem.
Ref: Using require.resolve to get resolved file path (node)
Jason's answer was the best answer, until Node.js ESM and the exports field came out.
Now that Node supports packages with an exports field that by default will prevent files like package.json from being resolvable unless the package author explicitly decides to expose them, the trick in Jason's answer will fail for packages that do not explicitly expose package.json.
There is a package called resolve-package-path that does the trick.
Here's how to use it:
const resolvePkg = require('resolve-package-path')
console.log(resolvePkg('#some/package'))
which will output something like
/path/to/#some/package/package.json
regardless of what the package's exports field contains.
I hope I correctly understand your needs: to get entry point file of some module. Let's say you want to get entry point of jugglingdb module:
node
> require('module')._resolveFilename('jugglingdb')
'/usr/local/lib/node_modules/jugglingdb/index.js'
As you can see this is not "official" way to get this kind of information about module, so behavior of this function may change from version to version. I've found it in node source: https://github.com/joyent/node/blob/master/lib/module.js#L280
According to #anatoliy solution, On MacOS X I have found the lookup paths doing
require('module')._resolveLookupPaths('myModule')
so I get the resolved lookup paths
[ 'myModule',
[ '/Users/admin/.node_modules',
'/Users/admin/.node_libraries',
'/usr/local/lib/node' ] ]
whereas the
require('module')._resolveFilename('myModule')
will not resolve the module I was looking for anyways, in fact the crazy thing is that the _load will not resolve the module:
> require('module')._load('myModule')
Error: Cannot find module 'myModule'
at Function.Module._resolveFilename (module.js:440:15)
at Function.Module._load (module.js:388:25)
at repl:1:19
at sigintHandlersWrap (vm.js:32:31)
at sigintHandlersWrap (vm.js:96:12)
at ContextifyScript.Script.runInContext (vm.js:31:12)
at REPLServer.defaultEval (repl.js:308:29)
at bound (domain.js:280:14)
at REPLServer.runBound [as eval] (domain.js:293:12)
at REPLServer.<anonymous> (repl.js:489:10)
while the require will:
> require('myModule')
but I don't have this module in
myProject/node_modules/
myProject/node_modules/#scope/
/usr/local/lib/node_modules/
/usr/local/lib/node_modules/#scope
/usr/local/lib/node_modules/npm/node_modules/
/usr/local/lib/node_modules/npm/node_modules/#scope
$HOME/.npm/
$HOME/.npm/#scope/
so where is this module???
First I had to do a $ sudo /usr/libexec/locate.updatedb
Then after some coffee I did locate myModule or better locate myModule/someFile.js
et voilà, it comes out that it was in a parent folder of my project i.e. outside my project root folder:
$pwd
/Users/admin/Projects/Node/myProject
$ ls ../../node_modules/myModule/
so you cannot avoid to rm -rf ../../node_modules/myModule/ and a fresh npm install.
I can argue that no one instructed npm to scan my computer in search for modules elsewhere than my project root folder where it was supposed to run or in the default modules search path.
This is maybe what you're looking for, check:
require.main.filename
Here is a solution that returns the module directory in a platform agnostic way. This does not use any 3rd party libraries and successfully locates ESM modules with "type": "module" and modules installed via npm link..
NOTE: If a particular module is a symlink to another location (eg. npm link) you will need use fs.realpath to get the location of the target directory:
const moduleDir = getModuleDir('some-npm-module');
const theRealPath = fs.realpathSync(moduleDir);
ESM
import fs from 'fs';
import path from 'path';
import { createRequire } from 'module';
/**
* Get's the file path to a module folder.
* #param {string} moduleEntry
* #param {string} fromFile
*/
const getModuleDir = (moduleEntry) => {
const packageName = moduleEntry.includes('/')
? moduleEntry.startsWith('#')
? moduleEntry.split('/').slice(0, 2).join('/')
: moduleEntry.split('/')[0]
: moduleEntry;
const require = createRequire(import.meta.url);
const lookupPaths = require.resolve.paths(moduleEntry).map((p) => path.join(p, packageName));
return lookupPaths.find((p) => fs.existsSync(p));
};
CommonJS
const fs = require('fs');
const path = require('path');
const { createRequire } = require('module');
/**
* Get's the file path to a module's folder.
* #param {string} moduleEntry
* #param {string} fromFile
*/
const getModuleDir = (moduleEntry, relativeToFile = __filename) => {
const packageName = moduleEntry.includes('/')
? moduleEntry.startsWith('#')
? moduleEntry.split('/').slice(0, 2).join('/')
: moduleEntry.split('/')[0]
: moduleEntry;
const require = createRequire(relativeToFile);
const lookupPaths = require.resolve.paths(moduleEntry).map((p) => path.join(p, packageName));
return lookupPaths.find((p) => fs.existsSync(p));
};
tl;dr : module resolution does not apply ?
Hello,
I am playing around with Typescript2 module resolution feature.
I've noticed that it is now possible to specify "Paths", so that you can do the following :
Old way
import {a} from "../../../foo"
New way
import {a} from "services/foo"
To do so, you need to add some configs to your tsconfig.json
"compilerOptions": {
"baseUrl": ".",
"paths": {
"services/*": ["./application/core/services/*"],
}
}
Problem that I have, is that when compiled, the import actually doesn't change. My javascript output still contains that import from "services/foo", so that obviously crash at runtime on my node server.
I use gulp-typescript to compile my javascript files :
var tsProject = ts.createProject("tsconfig.json");
return tsProject.src()
.pipe(sourcemaps.init())
.pipe(tsProject()).js
.pipe(sourcemaps.write("../api"))
.pipe(gulp.dest(function(file) {
return file.base;
}));
I am completely lost here and would love to use that module resolution, so that I can move away from that ../../ hell of imports. Any help would be more than appreciated !
The problem here is that the JavaScript Engine knows nothing about your TypeScript config, what you specify in your tsconfig is only used "compile time", when you have compiled your TypeScript into JS you need to do the same job as the TS compiler did but to save the resolved path in the JS file.
Simply put, all JS files needs to be processed and the aliases replaced with "real" paths.
Tip: Use the npm tool tspath (https://www.npmjs.com/package/tspath), it requires 0 configuration, just run it in somewhere in your project and all JS files will be processed and ready to run!
Is there a way to use Webpack loaders in a Node app / Run a Node app in a Webpack environment?
For instance I've got a webpack config that has a style-loader. In my Node app I do the following:
import style from 'style.css'
console.log(style.someClass)
I wanna run it like $ node app.js
I've got an idea that might work, based on the Webpack NodeJS API. What if we put the code that we want to be able to use the Webpack environment (with the configured module loaders) into a module:
appModule.js:
import style from 'style.css'
console.log(style.someClass)
And require it with the following:
app.js:
import Webpack from 'webpack'
import MemoryFS from 'memory-fs'
...
webpackConfig.entry = 'appModule.js'
webpackConfig.output = 'appModule-out.js'
let compiler = Webpack(webpackConfig)
let mfs = new MemoryFS()
compiler.outputFileSystem = mfs
compiler.run(function (err, stats) {
require(webpackConfig.output)
})
Probably it won't work because the require looks for the output on the physical FS... Can we require from the memory FS? I have not tried it yet - Any idea?
Webpack loaders aren't transpilers or interpreters, they simple gather assets that are then handled off to something like SASS or a text concatenator; within the confines of Webpacks environment.
Thus it is not possible to reuse them in the way you want, because while you can of course import and call them (they're still just functions + classes), they don't convert CSS to JSON objects (they don't do this) as you have written in your desired example.
It looks like you just need a JS implementation of a css parser - have a look at https://github.com/reworkcss/css
You should be able to create a compilation targeting the node environment which you can ultimately run by simply calling node output.js and this will immediately execute the entry point module.
Be aware, in case that you're using a newer version of Node.js, that Webpack doesn't support the ES2015 module syntax, so you'll have to configure Babel for Node.js as well to transform the modules.
I require a module that was installed via npm. I want to access a .js file subordinate to that module (so I can subclass a Constructor method in it). I can't (well, don't want to) modify the module's code, so don't have a place to extract its __dirname.
I am aware of the following question, but it is about getting the path of a module that one has code control over (hence, __dirname is the solution):
In Node.js how can I tell the path of `this` module?
~~~
Even better would be to get the module's loaded module info
If I correctly understand your question, you should use require.resolve():
Use the internal require() machinery to look up the location of a module, but rather than loading the module, just return the resolved filename.
Example: var pathToModule = require.resolve('module');
require.resolve() is a partial answer. The accepted answer may work for many node modules, but won't work for all of them.
require.resolve("moduleName") doesn't give you the directory where the module is installed; it gives you the location of the file defined in the main attribute in the module's package.json.
That might be moduleName/index.js or it could be moduleName/lib/moduleName.js. In the latter case, path.dirname(require.resolve("moduleName")) will return a directory you may not want or expect: node_modules/moduleName/lib
The correct way to get the complete path to a specific module is by resolving the filename:
let readmePath = require.resolve("moduleName/README.md");
If you just want the directory for the module (maybe you're going to make a lot of path.join() calls), then resolve the package.json — which must always be in the root of the project — and pass to path.dirname():
let packagePath = path.dirname(require.resolve("moduleName/package.json"));
FYI, require.resolve returns the module identifier according to CommonJS. In node.js this is the filename. In webpack this is a number.
In webpack situation, here is my solution to find out the module path:
const pathToModule = require.resolve('module/to/require');
console.log('pathToModule is', pathToModule); // a number, eg. 8
console.log('__webpack_modules__[pathToModule] is', __webpack_modules__[pathToModule]);
Then from __webpack_modules__[pathToModule] I got information like this:
(function(module, exports, __webpack_require__) {
eval("module.exports = (__webpack_require__(6))(85);\n\n//////////////////\n//
WEBPACK FOOTER\n// delegated ./node_modules/echarts/lib/echarts.js from dll-reference vendor_da75d351571a5de37e2e\n// module id = 8\n// module chunks = 0\n\n//# sourceURL=webpack:///delegated_./node_modules/echarts/lib/echarts.js_from_dll-reference_vendor_da75d351571a5de37e2e?");
/***/
})
Turned out I required old scripts from previous dll build file(for faster build speed), so that my updated module file didn't work as I expected. Finally I rebuilt my dll file and solved my problem.
Ref: Using require.resolve to get resolved file path (node)
Jason's answer was the best answer, until Node.js ESM and the exports field came out.
Now that Node supports packages with an exports field that by default will prevent files like package.json from being resolvable unless the package author explicitly decides to expose them, the trick in Jason's answer will fail for packages that do not explicitly expose package.json.
There is a package called resolve-package-path that does the trick.
Here's how to use it:
const resolvePkg = require('resolve-package-path')
console.log(resolvePkg('#some/package'))
which will output something like
/path/to/#some/package/package.json
regardless of what the package's exports field contains.
I hope I correctly understand your needs: to get entry point file of some module. Let's say you want to get entry point of jugglingdb module:
node
> require('module')._resolveFilename('jugglingdb')
'/usr/local/lib/node_modules/jugglingdb/index.js'
As you can see this is not "official" way to get this kind of information about module, so behavior of this function may change from version to version. I've found it in node source: https://github.com/joyent/node/blob/master/lib/module.js#L280
According to #anatoliy solution, On MacOS X I have found the lookup paths doing
require('module')._resolveLookupPaths('myModule')
so I get the resolved lookup paths
[ 'myModule',
[ '/Users/admin/.node_modules',
'/Users/admin/.node_libraries',
'/usr/local/lib/node' ] ]
whereas the
require('module')._resolveFilename('myModule')
will not resolve the module I was looking for anyways, in fact the crazy thing is that the _load will not resolve the module:
> require('module')._load('myModule')
Error: Cannot find module 'myModule'
at Function.Module._resolveFilename (module.js:440:15)
at Function.Module._load (module.js:388:25)
at repl:1:19
at sigintHandlersWrap (vm.js:32:31)
at sigintHandlersWrap (vm.js:96:12)
at ContextifyScript.Script.runInContext (vm.js:31:12)
at REPLServer.defaultEval (repl.js:308:29)
at bound (domain.js:280:14)
at REPLServer.runBound [as eval] (domain.js:293:12)
at REPLServer.<anonymous> (repl.js:489:10)
while the require will:
> require('myModule')
but I don't have this module in
myProject/node_modules/
myProject/node_modules/#scope/
/usr/local/lib/node_modules/
/usr/local/lib/node_modules/#scope
/usr/local/lib/node_modules/npm/node_modules/
/usr/local/lib/node_modules/npm/node_modules/#scope
$HOME/.npm/
$HOME/.npm/#scope/
so where is this module???
First I had to do a $ sudo /usr/libexec/locate.updatedb
Then after some coffee I did locate myModule or better locate myModule/someFile.js
et voilà, it comes out that it was in a parent folder of my project i.e. outside my project root folder:
$pwd
/Users/admin/Projects/Node/myProject
$ ls ../../node_modules/myModule/
so you cannot avoid to rm -rf ../../node_modules/myModule/ and a fresh npm install.
I can argue that no one instructed npm to scan my computer in search for modules elsewhere than my project root folder where it was supposed to run or in the default modules search path.
This is maybe what you're looking for, check:
require.main.filename
Here is a solution that returns the module directory in a platform agnostic way. This does not use any 3rd party libraries and successfully locates ESM modules with "type": "module" and modules installed via npm link..
NOTE: If a particular module is a symlink to another location (eg. npm link) you will need use fs.realpath to get the location of the target directory:
const moduleDir = getModuleDir('some-npm-module');
const theRealPath = fs.realpathSync(moduleDir);
ESM
import fs from 'fs';
import path from 'path';
import { createRequire } from 'module';
/**
* Get's the file path to a module folder.
* #param {string} moduleEntry
* #param {string} fromFile
*/
const getModuleDir = (moduleEntry) => {
const packageName = moduleEntry.includes('/')
? moduleEntry.startsWith('#')
? moduleEntry.split('/').slice(0, 2).join('/')
: moduleEntry.split('/')[0]
: moduleEntry;
const require = createRequire(import.meta.url);
const lookupPaths = require.resolve.paths(moduleEntry).map((p) => path.join(p, packageName));
return lookupPaths.find((p) => fs.existsSync(p));
};
CommonJS
const fs = require('fs');
const path = require('path');
const { createRequire } = require('module');
/**
* Get's the file path to a module's folder.
* #param {string} moduleEntry
* #param {string} fromFile
*/
const getModuleDir = (moduleEntry, relativeToFile = __filename) => {
const packageName = moduleEntry.includes('/')
? moduleEntry.startsWith('#')
? moduleEntry.split('/').slice(0, 2).join('/')
: moduleEntry.split('/')[0]
: moduleEntry;
const require = createRequire(relativeToFile);
const lookupPaths = require.resolve.paths(moduleEntry).map((p) => path.join(p, packageName));
return lookupPaths.find((p) => fs.existsSync(p));
};