I understand you can clear files from cache while using CommonJS simply by deleteing from require.cache and requireing the file again. However I've been unable to find an equivalent option when using ESM. Is this possible, and if so how?
Reading at esm issues it seems cache is intentionally not exposed.
I'm afraid the answer to your question is simply no.
it seems there is a solution using dynamic imports:
const modules = {
moduleA: async () => await import(`./module-a.js?v=${Date.now()}`)
}
Then use it like:
async function myTest() {
// module will be reset at each call of the function
const moduleA = await modules.moduleA()
}
Here is an issue about it with further details on that technique there
Related
Environment: webpack 5.44 + vue.js 3.0 + node 12.21
I'm trying to generate a module at compile-time, in order to avoid a costly computation at run-time (as well as 10Mb of dependencies that will never be used except during said computation). Basically run this at compile-time:
import * as BigModule from "big-module";
function extract_info(module) { ... }
export default extract_info(BigModule);
which will be imported at run-time as:
export default [ /* static info */ ];
I tried using val-loader (latest 4.0) which seems designed exactly for this use case.
Problem: big-module is an ESM, but val-loader apparently only supports CJS. So I can neither import ("Cannot use import statement outside a module" error) nor require ("Unexpected token 'export'" error).
Is there any way to make val-loader somehow load the ESM module? Note that I'm not bent on using val-loader, any other technique that achieves the same goal is just as welcome.
After learning way more than I wanted about this issue and node/webpack internals, there seems to be two possible approaches to import ESM from CJS:
Use dynamic import(). But it is asynchronous which makes it unfit here, as val-loader requires a synchronous result.
Transpile the ESM into CJS, which is the approach I took.
In my case, full transpiling is overkill and rewriting imports/exports is sufficient, so I'm using ascjs to rewrite the ESM files, along with eval to safely evaluate the resulting string.
All in all:
// require-esm.js
const fs = require('fs');
const ascjs = require('ascjs');
const _eval = require('eval');
function requireESM(file) {
file = require.resolve(file);
return _eval(ascjs(fs.readFileSync(file)), file, { require: requireESM }, true);
}
module.exports = requireESM;
// val-loader-target.js
const requireESM = require('./require-esm');
const BigModule = requireESM('big-module');
function extract_info(module) { ... }
module.exports = extract_info(BigModule);
Note that:
ascjs is safe to use on CJS modules, since it only rewrites ESM imports/exports. So it's OK for big-module or its dependencies to require CJS files.
the third argument to _eval enables recursive rewriting, otherwise only the top-level file (the one passed to requireESM) is translated.
app.js
require('./modules/mod');
modules/mod/mod.js
modules.exports = () => {
require('./modules/secondmodule');
}
Essentially I want the above code to be able to require modules, but using the same context that itself was called from, e.g. another module in the same folder, without having to use relative paths.
I thought module.require() did this, but it seems to give me the same error that require() was after I moved my code into the separate module (mod.js).
edit:
I have since discovered I can use require.parent.module and it seems to be working. Please let me know if this is not advised.
require uses paths that are relative to current module. It's possible to do this by providing require from parent module:
modules.exports = parentRequire => {
parentRequire('./modules/secondmodule');
}
Which is used like:
require('./modules/mod')(require);
It's correct to use relative modules instead because child module shouldn't be aware of the context in which it's evaluated:
require('../secondmodule');
In case mod has to be decoupled from secondmodule, dependencies can be provided to it with some common pattern, e.g. dependency injection or service locator.
Secondary optional answer:
module.exports = () => {
module.parent.require('./modules/secondmodule');
}
I trying to convert a nodejs project to TypeScript and while mostly I did not faced really difficult obstacles during this process, the codebase has few gotchas like this, mostly in startup code:
function prepareConfiguration() {
let cloudConfigLoader = require('../utils/cloud-config');
return cloudConfigLoader.ensureForFreshConfig().then(function() {
//do some stuff
});
}
I may be need just an advice on which approach for refactoring this has less code changes to be made to make it work in TypeScript fashion.
In response to comments, more details:
That require loads the node module, not a JSON file. From that module the ensureForFreshConfig function contacts with a cloud service to load a list of values to rebuild a configuration state object.
Problem is that mdule was made in standard node approach of "module is isngleton object" and its independencies include auth component that will be ready only when the shown require call is made. I know it is not best a way to do so..
Typesript does not allow "mport that module later" except with dynamyc import which is problematic as mentiond in comment.
The "good" approach is to refactor that part of startup and make the ensureForFreshConfig and its dependency to initiate its ntenras on demand via cnstructors.. I just hoped ofr some soluiton to reduce things to be remade durng this transition to the TypeScript
import { cloudConfigLoader } from '../utils/cloud-config'
async function prepareConfiguration() {
await cloudConfigLoader.ensureForFreshConfig()
// do some stuff
// return some-stuff
}
The function is to be used as follows.
await prepareConfiguration()
I am building a project with Electron, and using Webpack to build the (Angular 2) render process app.
In this app, I need to dynamically require some files at run-time which do not exist at build-time. The code looks something like this:
require("fs").readdirSync(this.path).forEach(file => {
let myModule = require(path.join(this.path, file));
// do stuff with myModule
});
The problem is that the Webpack compiler will convert the require() call to its own __webpack_require__() and at run-time, it will look in its own internal module registry for the dynamic "myModule" file, and of course will not find it.
I have tried using the "externals" config option, but since this is a dynamic require, it doesn't seem to be processed by "externals".
Anyone else had success in solving this problem?
As suggested in a comment to my question by #jantimon, the solution is to use global.require:
require("fs").readdirSync(this.path).forEach(file => {
let myModule = global.require(path.join(this.path, file));
// do stuff with myModule
});
I came across this article and for some other reason the author needs node modules which gets not transpiled by webpack. He suggested to use
new webpack.IgnorePlugin(new RegExp("^(fs|ipc)$"))
in the webpack.config.js file. This should prevent to transpile the module fs and ipc so it can be used (required) in code.
I am not really sure if this hits also your problem but it might help.
The original article for more context can be found here: https://medium.com/#Agro/developing-desktop-applications-with-electron-and-react-40d117d97564#.927tyjq0y
I'm testing a bunch of React JSX components. They all need to be transpiled with React, or Babel or whatever, but we have special needs for stubbing requirements, so I'm trying to override requires with a special compiler that's run with Mocha. The solution below works well, but you'll notice that we're using require.extensions[] to capture all the .jsx files. What concerns me is that require.extensions is locked and deprecated. Is there any better way to do this?
// Install the compiler.
require.extensions['.jsx'] = function(module, filename) {
return module._compile(transform(filename), filename);
};
Here's the whole transpiler for reference:
// Based on https://github.com/Khan/react-components/blob/master/test/compiler.js
var fs = require('fs'),
ReactTools = require('react-tools');
// A module that exports a single, stubbed-out React Component.
var reactStub = 'module.exports = require("react").createClass({render:function(){return null;}});';
// Should this file be stubbed out for testing?
function shouldStub(filename) {
if (!global.reactModulesToStub) return false;
// Check if the file name ends with any stub path.
var stubs = global.reactModulesToStub;
for (var i = 0; i < stubs.length; i++) {
if (filename.substr(-stubs[i].length) == stubs[i]) {
console.log('should stub', filename);
return true;
}
}
return false;
}
// Transform a file via JSX/Harmony or stubbing.
function transform(filename) {
if (shouldStub(filename)) {
delete require.cache[filename];
return reactStub;
} else {
var content = fs.readFileSync(filename, 'utf8');
return ReactTools.transform(content, {harmony: true});
}
}
// Install the compiler.
require.extensions['.jsx'] = function(module, filename) {
return module._compile(transform(filename), filename);
};
And some links to simalar solutions...
https://github.com/danvk/mocha-react/issues/1
https://github.com/Automattic/jsx-require-extension
https://www.npmjs.com/package/node-jsx
https://github.com/olalonde/better-require
http://mochajs.org/#usage
http://nodejs.org/api/globals.html#globals_require_extensions
A solution can be forked from here:
https://github.com/danvk/mocha-react
There are two reasons that API has been deprecated. One, the node module resolution algorithm is VERY complicated, it has to look at the specified file, if it doesn't exist it looks for that file and all the possible extensions in the keys of require.extensions, and if it's a directory, look for a package.json or index.js. Oh, and don't forget, if there is no ./ at the beginning, it looks in the node_modules directory, looking at the parent directory if it can't be found in that node_modules. Ryan Dahl said he regrets making it so complicated in his talk at JsConf 2018, and uses a much simpler module resolution algorithm in his deno project. Two, it needs more filesystem calls if there are more extensions in require.extensions, because it has to match extensionless files.
A solution to the second problem is require-extension. I haven't used it myself, but it abstracts the require.extensions API and makes it much more performant.
There is no other way to do this, and this is how everybody does transpiling (babel, etc). #uni_nake's answer - to use node-hook - is OK in that it hides this from you, but it essentially uses the same mechanism: a look in it's code shows that it uses Module._extensions, but this is the same as require.extensions, as shown by a test I wrote: https://github.com/giltayar/playing/blob/1f04f6ddc1a0028974b403b4d1974afa872edba4/javascript/node/test/is-module-extensions-same-as-require-extensions.test.js
So final answer - I would assume that nobody at Node will break Babel, and if they do, they will probably give another solution for the same problem. I would not hesitate to use it!
I use node-hook to stub all .scss calls in my tests.
You'll see from the docs that when a matching file is required it will execute the containing string instead and is really quite powerful as it also passes you the original source.
Hope that is what you're looking for.
I think you should use pirates
I think the PR when require.extensions used in babel-register was replaced with pirates would be helpful.
https://github.com/babel/babel/pull/3670/files#diff-75a0292ed78043766c2d5564edd84ad2L85-L93
Hope that is what you're looking for.