when you add imports to an ESM package, you can resolve alias paths into the corresponding real ones, like this:
package.json
{
"type": "module",
"imports": { "#*": "./*" }
...
}
now we can use it like this import something from "#module" which resolves into import something from "./module"
but, I can't see any docs about using the same resolving mechanism for builtin functions (or maybe user functions either) that accept PathLike arguments like readFileSync
readFileSync("#file.txt")
I know it is a module resolving, but we can use it as a path resolving.
my question is: how to achieve the same mechanism to resolve any PathLike path, without a manual modification, because simply we need to dispense the relative paths
an example of manual resolving
function example(path: PathLike){
let imports = /* read the property imports from package.json */
let keys = Object.keys(imports)
// make a loop, and for each key make something like this (needs additional work)
let resolvedPath = path.replace('#','./')
// now, use resolvedPath instead of path
}
example of nodejs resolving
function example2(path: PathLike){
let resolvedPath = resolve(path)
}
in addition to the original work of resolve() it also uses the property import to resolve the path to avoid using relative paths
I ask if there is a way to achieve this goal.
The solution
thanks to #RickN this is the final solution:
export function resolveImports(path: PathLike): Promise<string> {
return import.meta!.resolve!(path.toString()).then((resolvedPath) =>
fileURLToPath(resolvedPath)
);
}
As of writing, you need a CLI flag to enable the resolve method in import.meta:
node --experimental-import-meta-resolve your-file.js
(The first part of the name speaks for itself.)
Then, in a script:
console.log( await import.meta.resolve('#foo/bar.js') );
// It can resolve any type of file, even if you can't import() it.
console.log( await import.meta.resolve('#foo/hello.txt') );
As it's a promise-based function, you'll need to await the result.
async function example2(path: PathLike){
return import.meta.resolve(path);
}
As a side-note: this returns a URL (file:///...) so run the result through the built-in URL class url.fileURLToPath() if you just want a file path:
import { fileURLToPath } from 'node:url';
// OR: const { fileURLToPath } = require('node:url');
// For older node versions use `require('url')`
console.log( fileURLToPath(await import.meta.resolve('#foo/hello.txt')) );
// => /tmp/example/directory-with-modules-in-it/hello.txt
Related
From the node.js documentation:
Modules are cached after the first time they are loaded. This means (among other things) that every call to require('foo') will get exactly the same object returned, if it would resolve to the same file.
Is there a way to invalidate this cache? i.e. for unit testing, I'd like each test to be working on a fresh object.
You can always safely delete an entry in require.cache without a problem, even when there are circular dependencies. Because when you delete, you just delete a reference to the cached module object, not the module object itself, the module object will not be GCed because in case of circular dependencies, there is still a object referencing this module object.
Suppose you have:
script a.js:
var b=require('./b.js').b;
exports.a='a from a.js';
exports.b=b;
and script b.js:
var a=require('./a.js').a;
exports.b='b from b.js';
exports.a=a;
when you do:
var a=require('./a.js')
var b=require('./b.js')
you will get:
> a
{ a: 'a from a.js', b: 'b from b.js' }
> b
{ b: 'b from b.js', a: undefined }
now if you edit your b.js:
var a=require('./a.js').a;
exports.b='b from b.js. changed value';
exports.a=a;
and do:
delete require.cache[require.resolve('./b.js')]
b=require('./b.js')
you will get:
> a
{ a: 'a from a.js', b: 'b from b.js' }
> b
{ b: 'b from b.js. changed value',
a: 'a from a.js' }
===
The above is valid if directly running node.js. However, if using tools that have their own module caching system, such as jest, the correct statement would be:
jest.resetModules();
If you always want to reload your module, you could add this function:
function requireUncached(module) {
delete require.cache[require.resolve(module)];
return require(module);
}
and then use requireUncached('./myModule') instead of require.
Yes, you can access the cache via require.cache[moduleName] where moduleName is the name of the module you wish to access. Deleting an entry by calling delete require.cache[moduleName] will cause require to load the actual file.
This is how you would remove all cached files associated with the module:
/**
* Removes a module from the cache
*/
function purgeCache(moduleName) {
// Traverse the cache looking for the files
// loaded by the specified module name
searchCache(moduleName, function (mod) {
delete require.cache[mod.id];
});
// Remove cached paths to the module.
// Thanks to #bentael for pointing this out.
Object.keys(module.constructor._pathCache).forEach(function(cacheKey) {
if (cacheKey.indexOf(moduleName)>0) {
delete module.constructor._pathCache[cacheKey];
}
});
};
/**
* Traverses the cache to search for all the cached
* files of the specified module name
*/
function searchCache(moduleName, callback) {
// Resolve the module identified by the specified name
var mod = require.resolve(moduleName);
// Check if the module has been resolved and found within
// the cache
if (mod && ((mod = require.cache[mod]) !== undefined)) {
// Recursively go over the results
(function traverse(mod) {
// Go over each of the module's children and
// traverse them
mod.children.forEach(function (child) {
traverse(child);
});
// Call the specified callback providing the
// found cached module
callback(mod);
}(mod));
}
};
Usage would be:
// Load the package
var mypackage = require('./mypackage');
// Purge the package from cache
purgeCache('./mypackage');
Since this code uses the same resolver require does, just specify whatever you would for require.
"Unix was not designed to stop its users from doing stupid things, as
that would also stop them from doing clever things." – Doug Gwyn
I think that there should have been a way for performing an explicit uncached module loading.
There's a Simple Module for that (with tests)
We had this exact issue while testing our code (delete cached modules so they can be re-required in a fresh state) so we reviewed all the suggestions of people on the various StackOverflow Questions & Answers and put together a simple node.js module (with tests):
https://www.npmjs.com/package/decache
As you would expect, works for both published npm packages and locally defined modules. Windows, Mac, Linux, etc.
How? (usage)
Usage is pretty simple:
install
Install the module from npm:
npm install decache --save-dev
Use it in your code:
// require the decache module:
const decache = require('decache');
// require a module that you wrote"
let mymod = require('./mymodule.js');
// use your module the way you need to:
console.log(mymod.count()); // 0 (the initial state for our counter is zero)
console.log(mymod.incrementRunCount()); // 1
// delete the cached module:
decache('./mymodule.js');
//
mymod = require('./mymodule.js'); // fresh start
console.log(mymod.count()); // 0 (back to initial state ... zero)
If you have any questions or need more examples, please create a GitHub issue:
https://github.com/dwyl/decache/issues
For anyone coming across this who is using Jest, because Jest does its own module caching, there's a built-in function for this - just make sure jest.resetModules runs eg. after each of your tests:
afterEach( function() {
jest.resetModules();
});
Found this after trying to use decache like another answer suggested. Thanks to Anthony Garvan.
Function documentation here.
The solutions is to use:
delete require.cache[require.resolve(<path of your script>)]
Find here some basic explanations for those who, like me, are a bit new in this:
Suppose you have a dummy example.js file in the root of your directory:
exports.message = "hi";
exports.say = function () {
console.log(message);
}
Then you require() like this:
$ node
> require('./example.js')
{ message: 'hi', say: [Function] }
If you then add a line like this to example.js:
exports.message = "hi";
exports.say = function () {
console.log(message);
}
exports.farewell = "bye!"; // this line is added later on
And continue in the console, the module is not updated:
> require('./example.js')
{ message: 'hi', say: [Function] }
That's when you can use delete require.cache[require.resolve()] indicated in luff's answer:
> delete require.cache[require.resolve('./example.js')]
true
> require('./example.js')
{ message: 'hi', say: [Function], farewell: 'bye!' }
So the cache is cleaned and the require() captures the content of the file again, loading all the current values.
rewire is great for this use case, you get a new instance with each call. Easy dependency injection for node.js unit testing.
rewire adds a special setter and getter to modules so you can modify their behaviour for better unit testing. You may
inject mocks for other modules or globals like process
leak private variables
override variables within the module.
rewire does not load the file and eval the contents to emulate node's require mechanism. In fact it uses node's own require to load the module. Thus your module behaves exactly the same in your test environment as under regular circumstances (except your modifications).
Good news to all caffeine-addicts: rewire works also with Coffee-Script. Note that in this case CoffeeScript needs to be listed in your devDependencies.
I'd add to luff's answer one more line and change the parameter name:
function requireCached(_module){
var l = module.children.length;
for (var i = 0; i < l; i++)
{
if (module.children[i].id === require.resolve(_module))
{
module.children.splice(i, 1);
break;
}
}
delete require.cache[require.resolve(_module)];
return require(_module)
}
Yes, you can invalidate cache.
The cache is stored in an object called require.cache which you can access directly according to filenames (e.g. - /projects/app/home/index.js as opposed to ./home which you would use in a require('./home') statement).
delete require.cache['/projects/app/home/index.js'];
Our team has found the following module useful. To invalidate certain groups of modules.
https://www.npmjs.com/package/node-resource
I am not 100% certain of what you mean by 'invalidate', but you can add the following above the require statements to clear the cache:
Object.keys(require.cache).forEach(function(key) { delete require.cache[key] })
Taken from #Dancrumb's comment here
requireUncached with relative path: 🔥
const requireUncached = require => module => {
delete require.cache[require.resolve(module)];
return require(module);
};
module.exports = requireUncached;
invoke requireUncached with relative path:
const requireUncached = require('../helpers/require_uncached')(require);
const myModule = requireUncached('./myModule');
I couldn't neatly add code in an answer's comment. But I would use #Ben Barkay's answer then add this to the require.uncache function.
// see https://github.com/joyent/node/issues/8266
// use in it in #Ben Barkay's require.uncache function or along with it. whatever
Object.keys(module.constructor._pathCache).forEach(function(cacheKey) {
if ( cacheKey.indexOf(moduleName) > -1 ) {
delete module.constructor._pathCache[ cacheKey ];
}
});
Say you've required a module, then uninstalled it, then reinstalled the same module but used a different version that has a different main script in its package.json, the next require will fail because that main script does not exists because it's cached in Module._pathCache
If you want a module to simply never be cached (sometimes useful for development, but remember to remove it when done!) you can just put delete require.cache[module.id]; inside the module.
here's my version of this answer, which handles not loading a file if it has (for example) syntax errors
function reacquire(module) {
const fullpath = require.resolve(module);
const backup = require.cache[fullpath];
delete require.cache[fullpath];
try {
const newcopy = require(module);
console.log("reqcquired:",module,typeof newcopy);
return newcopy;
} catch (e) {
console.log("Can't reqcquire",module,":",e.message);
require.cache[fullpath] = backup;
return backup;
}
}
Following two step procedure is working perfectly for me.
After changing Model file i-e 'mymodule.js' dynamically, you need to Delete precompiled model in mongoose model first then reload it using require-reload
Example:
// Delete mongoose model
delete mongoose.connection.models[thisObject.singular('mymodule')]
// Reload model
var reload = require('require-reload')(require);
var entityModel = reload('./mymodule.js');
The documentation says:
Modules are cached in this object when they are required. By deleting a key value from this object, the next require will reload the module. This does not apply to native addons, for which reloading will result in an error.
If it's for unit tests, another good tool to use is proxyquire. Everytime you proxyquire the module, it will invalidate the module cache and cache a new one. It also allows you to modify the modules required by the file that you are testing.
I made a small module to delete module from the cache after loading. This forces reevaluation of the module next time it is required. See https://github.com/bahmutov/require-and-forget
// random.js
module.exports = Math.random()
const forget = require('require-and-forget')
const r1 = forget('./random')
const r2 = forget('./random')
// r1 and r2 will be different
// "random.js" will not be stored in the require.cache
PS: you can also put "self-destruct" into the module itself. See https://github.com/bahmutov/unload-me
PSS: more tricks with Node require in my https://glebbahmutov.com/blog/hacking-node-require/
I have an external library that is exported as a function, in the Stub documentation it only accepts an input with the first parameter as object and the second parameter as method , so how could I stub a library that is exported as a function in a Node ES Modules environment (without Commonjs)?
(In my specific case, I had used a library that use the internet to work, and I wanted to test derivated functions without accessing the internet, so I want to stub the external function library)
Attempts:
I couldn't use solutions like proxyquire as it is a solution based on require and module cache deletion, which are not supported within Node's ES modules.
I don't want to use proxyquire-universal because it simulates the operation of require in normal ES, and it's just a function in the whole project that I wanted to test, I was looking for a simpler solution
Changing the import mode doesn't work as it's not recompiled like in babel, so even if I import as import * as obj from 'lib' only the function name is changed
I had this error environment with 3 files:
An external library called "sum" for example, which I don't want to change, exported as follows:
External Library: externalSum.js
module.exports = function sum(a, b){
console.log(">>>>> running without stub <<<<<")
return a + b
}
This library used in the middle of a project file called mathProblems
Internal File: mathProblems.js
import sum from 'externalSum'
export function sumMore1(a) {
return sum(a, 1);
}
And I have a test file
Internal File: spec.js
import sinon from 'sinon'
import assert from 'assert'
import sumObj from 'externalSum'
import { sumMore1 } from '../mathProblems.js'
describe('sumMore1 is working', () => {
it('sumMore1 test', () => {
const sum_stub = sinon.stub(sumObj) // NOT STUBBING
sum_stub.withArgs(1, 1).returns(2) // NOT STUBBING
const result = sumMore1(1)
assert.strictEqual(result, 2)
});
});
I didn't find this solution anywhere on the internet, i found some solutions that work for node with request or babilon, but not for my case using ES Modules:
https://github.com/sinonjs/sinon/issues/562
https://minaluke.medium.com/how-to-stub-spy-a-default-exported-function-a2dc1b580a6b
So I wanted to register the solution in case anyone needs it.
To solve this, create a new file, which can be allocated anywhere in the project, in this case I will call it sumImport.js:
Internal File: sumImport.js
import sum from 'externalSum';
// export as object
export default {
sum
}
The object needs to be called inside the created function I want to test, and changed the import way:
Internal File: mathProblems.js
import sumObj from './sumImport.js';
export function sumMore1(a) {
const { sum } = sumObj;
return sum(a, 1);
}
And I finally managed to import as an object in the test:
Internal File: spec.js
import sinon from 'sinon'
import assert from 'assert'
import sumObj from '../sumImport.js'
import { sumMore1 } from '../mathProblems.js'
describe('sumMore1 is working', () => {
it('sumMore1 test', () => {
const sum_stub = sinon.stub(sumObj, "sum") // STUBBED
sum_stub.withArgs(1, 1).returns(2) // STUBBED
const result = sumMore1(1)
assert.strictEqual(result, 2)
});
});
I hope it helps someone and if someone else has some better solutions I would also be grateful!
For example, when I need a module located inside node_modules I can write const module = require('module') or import module from 'module'. If the module located in another location I need to write const module = require('../location/module'), etc...
This behavior not always desired, for example, If I have examples that I want a user will be able to copy/paste and run, and also to run this example from the project folder itself.
Can this be achieved?
I don't understand why I get downvoted, but I ended up to write the following wrapper which does the job:
const customRequire = function (moduleName, path) {
let module
try {
module = require(moduleName)
} catch (e) {
try {
module = require(`${path}/${moduleName}`)
} catch (e1) {
throw Error(`module ${moduleName} on path ${path} wasn't found`)
}
}
return module
}
Now if you copy/paste this code it will work. And if you will run it within the project where required modules defined and not located in node_modules it will work too.
The solution above work for the CommonJS module system, for ES6 you need to use the function import, to dynamically import modules
import('/modules/my-module.js')
.then((module) => {
// Do something with the module.
});
I'm using AVA + sinon to build my unit test. Since I need ES6 modules and I don't like babel, I'm using mjs files all over my project, including the test files. I use "--experimental-modules" argument to start my project and I use "esm" package in the test. The following is my ava config and the test code.
"ava": {
"require": [
"esm"
],
"babel": false,
"extensions": [
"mjs"
]
},
// test.mjs
import test from 'ava';
import sinon from 'sinon';
import { receiver } from '../src/receiver';
import * as factory from '../src/factory';
test('pipeline get called', async t => {
const stub_factory = sinon.stub(factory, 'backbone_factory');
t.pass();
});
But I get the error message:
TypeError {
message: 'ES Modules cannot be stubbed',
}
How can I stub an ES6 module without babel?
According to John-David Dalton, the creator of the esm package, it is only possible to mutate the namespaces of *.js files - *.mjs files are locked down.
That means Sinon (and all other software) is not able to stub these modules - exactly as the error message points out. There are two ways to fix the issue here:
Just rename the files' extension to .js to make the exports mutable. This is the least invasive, as the mutableNamespace option is on by default for esm. This only applies when you use the esm loader, of course.
Use a dedicated module loader that proxies all the imports and replaces them with one of your liking.
The tech stack agnostic terminology for option 2 is a link seam - essentially replacing Node's default module loader. Usually one could use Quibble, ESMock, proxyquire or rewire, meaning the test above would look something like this when using Proxyquire:
// assuming that `receiver` uses `factory` internally
// comment out the import - we'll use proxyquire
// import * as factory from '../src/factory';
// import { receiver } from '../src/receiver';
const factory = { backbone_factory: sinon.stub() };
const receiver = proxyquire('../src/receiver', { './factory' : factory });
Modifying the proxyquire example to use Quibble or ESMock (both supports ESM natively) should be trivial.
Sinon needs to evolve with the times or be left behind (ESM is becoming defacto now with Node 12) as it is turning out to be a giant pain to use due to its many limitations.
This article provides a workaround (actually 4, but I only found 1 to be acceptable). In my case, I was exporting functions from a module directly and getting this error: ES Modules cannot be stubbed
export function abc() {
}
The solution was to put the functions into a class and export that instead:
export class Utils {
abc() {
}
}
notice that the function keyword is removed in the method syntax.
Happy Coding - hope Sinon makes it in the long run, but it's not looking good given its excessive rigidity.
Sticking with the questions Headline „Stub an export from a native ES Module without babel“ here's my take, using mocha and esmock:
(credits: certainly #oligofren brought me on the right path…)
package.json:
"scripts": {
...
"test": "mocha --loader=esmock",
"devDependencies": {
"esmock": "^2.1.0",
"mocha": "^10.2.0",
TestDad.js (a class)
import { sonBar } from './testSon.js'
export default class TestDad {
constructor() {
console.log(purple('constructing TestDad, calling...'))
sonBar()
}
}
testSon.js (a 'util' library)
export const sonFoo = () => {
console.log(`Original Son 'foo' and here, my brother... `)
sonBar()
}
export const sonBar = () => {
console.log(`Original Son bar`)
}
export default { sonFoo, sonBar }
esmockTest.js
import esmock from 'esmock'
describe.only(autoSuiteName(import.meta.url),
() => {
it('Test 1', async() => {
const TestDad = await esmock('../src/commands/TestDad.js', {
'../src/commands/testSon.js': {
sonBar: () => { console.log('STEPSON Bar') }
}
})
// eslint-disable-next-line no-new
new TestDad()
})
it('Test 2', async() => {
const testSon = await esmock('../src/commands/testSon.js')
testSon.sonBar = () => { console.log('ANOTHER STEPSON Bar') }
testSon.sonFoo() // still original
testSon.sonBar() // different now
})
})
autoSuiteName(import.meta.url)
regarding Test1
working nicely, import bended as desired.
regarding Test1
Bending a single function to do something else is not a problem.
(but then there is not much test value in calling your very own function you just defined, is there?)
Enclosed function calls within the module (i.e. from sonFoo to sonBar) remain what they are, they are indeed a closure, still pointing to the prior function
Btw also tested that: No better results with sinon.callsFake() (would have been surprising if there was…)
I have node.js application written in TypeScript and I need to switch between two interface implementations based on the config file. Currently I have this code which seems to be working.
"use strict";
import { config } from "../config";
let Something;
if (config.useFakeSomething) {
Something = require("./FakeSomething").FakeSomething;
} else {
Something = require("./RealSomething").RealSomething;
}
...
let s = new Something();
s.DoStuff();
...
But I have bad gut feeling about that (mainly because of the mixing require and import for loading modules). Is there some other way how to achieve implementation switching based on config file without importing both modules?
If you want to keep the client-code for your Something class clean, you can move the conditional importing to a single file. You can have the following directory structure for your Something module:
/Something
RealSomething.ts
FakeSomething.ts
index.ts
And in your index.ts, you can have the following:
import { config } from '../config';
const Something = config.useFakeSomething ?
require('./FakeSomething').FakeSomething :
require('./RealSomething').RealSomething;
export default Something;
And in your client code, you can just import Something:
import Something from './Something/index';
I can see nothing wrong with your approach. In fact lines like
import { config } from "../config";
When targeting commonjs will be compiled to the following javascript (ES6):
const config = require('../config');
So they are effectively identical and you are not mixing different module loading techniques.
You can do it like that:
let moduleLoader:any;
if( pladform == 1 ) {
moduleLoader = require('./module1');
} else {
moduleLoader = require('./module2');
}
and then
if( pladform === 1 ) {
platformBrowserDynamic().bootstrapModule(moduleLoader.module1, [ languageService ]);
}
else if ( pladform === 2 ) {
platformBrowserDynamic().bootstrapModule(moduleLoader.module2, [ languageService ]);
}
In addition to the correct answers above, in case you need this switching for many files within a single folder, you can use a symbolic-link (not available on Windows) that referenced to the right folder, so your code remains clean.
This approach is good for switching between real code and stubs, for instance
A modern answer using the import() function:
import { config } from "../config";
let Something;
if (config.useFakeSomething) {
Something = (await import("./FakeSomething")).FakeSomething;
} else {
Something = (await import("./RealSomething")).RealSomething;
}
...
let s = new Something();
s.DoStuff();
...
Remember that import() is non blocking, so you need to add await if the code that follows needs the result from the import().