My title is a bit vague, here is what I'm trying to do:
I have a typescript npm package
I want it to be useable on both node and browser.
I'm building it using a simple tsc command (no bundling), in order to get proper typings
My module has 1 entry point, an index.ts file, which exposes (re-exports) everything.
Some functions in this module are meant to be used on node-only, so there are node imports in some files, like:
import { fileURLToPath } from 'url'
import { readFile } from 'fs/promises'
import { resolve } from 'path'
// ...
I would like to find a way to:
Not trip-up bundlers with this
Not force users of this package to add "hacks" to their bundler config, like mentioned here: Node cannot find module "fs" when using webpack
Throw sensible Errors in case they are trying to use node-only features
Use proper typings inside my module, utilizing #types/node in my code
My main problem is, that no matter what, I have to import or require the node-only modules, which breaks requirement 1 (trips up bundlers, or forces the user to add some polyfill).
The only way I found that's working, is what isomorphic packages use, which is to have 2 different entry points, and mark it in my package.json like so:
{
// The entry point for node modules
"main": "lib/index.node.js",
// The entry point for bundlers
"browser": "lib/index.browser.js",
// Common typings
"typings": "lib/index.browser.d.ts"
}
This is however very impractical, and forces me to do a lots of repetition, as I don't have 2 different versions of the package, just some code that should throw in the browser when used.
Is there a way to make something like this work?
// create safe-fs.ts locally and use it instead of the real "fs" module
import * as fs from 'fs'
function createModuleProxy(moduleName: string): any {
return new Proxy(
{},
{
get(target, property) {
return () => {
throw new Error(`Function "${String(property)}" from module "${moduleName}" should only be used on node.js`)
}
},
},
)
}
const isNode = typeof window === undefined && typeof process === 'object'
const safeFs: typeof fs = isNode ? fs : createModuleProxy('fs')
export default safeFs
As it stands, this trips up bundlers, as I'm still importing fs.
Related
I was trying to run some NodeJS code using QickJS and NectarJS, and I stumble in an old question I have, how to import modules.
In special, qjs -m <source-js> loads ES6 modules, and expect import export statements,, instead of require, and writing module.exports, or exports.
Also I use an builtin std module (that cannot be resolved by node_module).
For the first example I changed the code manually, but I would like to have a script to do the transformations. I expected this to be doable with webpack, but I can't find how.
Example
main.js
const h = require('./hello.js');
h.sayHello();
hello.js
const h = require('./print.js');
function sayHello(){
h.print("Hello");
}
module.exports = {sayHello};
print.js
module.exports = {
print(s) {
console.log(s);
}
}
I can run this with the command node main.js, but qjs -m main.js, will fail with 'require' is not defined
I am currently having a TypeScript project that includes Ethers.js, which in turn includes bn.js.
The problem is
SyntaxError: The requested module './../../../bn.js/lib/bn.js' does not provide an export named 'default'
It appears to me this is because BN is in UMD format (see at https://github.com/indutny/bn.js/blob/master/lib/bn.js#L1)
(function (module, exports) {
'use strict';
// Utils
function assert (val, msg) {
if (!val) throw new Error(msg || 'Assertion failed');
}
and the correponding .ts declaration is
"use strict";
/**
* BigNumber
*
* A wrapper around the BN.js object. We use the BN.js library
* because it is used by elliptic, so it is required regardless.
*
*/
import _BN from "bn.js";
import BN = _BN.BN;
import { Bytes, Hexable, hexlify, isBytes, isHexString } from "#ethersproject/bytes";
import { Logger } from "#ethersproject/logger";
import { version } from "./_version";
const logger = new Logger(version);
It may be there's something that could be done at importing (source) or in Rollup. Difficult to tell!
Here is a screenshot of the build errors (one variation, depending on if building or running directly)
Question: Is there a way to transform this format to an ESM format in application Rollup pipeline?
I have tried using #rollup/plugin-commonjs and #rollup/plugin-node-resolve as in
resolve({ browser: true, preferBuiltins: false }), commonjs()]
(or see the project as whole at https://github.com/veikkoeeva/erc1155sample/blob/main/web/rollup.config.js, the error shows with npm run test or npm run start (in console log)).
Thus far I've had no luck cracking this, though. Hence coming here wondering if there's a dumb issue I don't see or if this is a genuinely tougher issue.
Edit: indeed, following https://rollupjs.org/guide/en/#error-name-is-not-exported-by-module and maybe named exports is the key here...
For example, when I need a module located inside node_modules I can write const module = require('module') or import module from 'module'. If the module located in another location I need to write const module = require('../location/module'), etc...
This behavior not always desired, for example, If I have examples that I want a user will be able to copy/paste and run, and also to run this example from the project folder itself.
Can this be achieved?
I don't understand why I get downvoted, but I ended up to write the following wrapper which does the job:
const customRequire = function (moduleName, path) {
let module
try {
module = require(moduleName)
} catch (e) {
try {
module = require(`${path}/${moduleName}`)
} catch (e1) {
throw Error(`module ${moduleName} on path ${path} wasn't found`)
}
}
return module
}
Now if you copy/paste this code it will work. And if you will run it within the project where required modules defined and not located in node_modules it will work too.
The solution above work for the CommonJS module system, for ES6 you need to use the function import, to dynamically import modules
import('/modules/my-module.js')
.then((module) => {
// Do something with the module.
});
I'm using AVA + sinon to build my unit test. Since I need ES6 modules and I don't like babel, I'm using mjs files all over my project, including the test files. I use "--experimental-modules" argument to start my project and I use "esm" package in the test. The following is my ava config and the test code.
"ava": {
"require": [
"esm"
],
"babel": false,
"extensions": [
"mjs"
]
},
// test.mjs
import test from 'ava';
import sinon from 'sinon';
import { receiver } from '../src/receiver';
import * as factory from '../src/factory';
test('pipeline get called', async t => {
const stub_factory = sinon.stub(factory, 'backbone_factory');
t.pass();
});
But I get the error message:
TypeError {
message: 'ES Modules cannot be stubbed',
}
How can I stub an ES6 module without babel?
According to John-David Dalton, the creator of the esm package, it is only possible to mutate the namespaces of *.js files - *.mjs files are locked down.
That means Sinon (and all other software) is not able to stub these modules - exactly as the error message points out. There are two ways to fix the issue here:
Just rename the files' extension to .js to make the exports mutable. This is the least invasive, as the mutableNamespace option is on by default for esm. This only applies when you use the esm loader, of course.
Use a dedicated module loader that proxies all the imports and replaces them with one of your liking.
The tech stack agnostic terminology for option 2 is a link seam - essentially replacing Node's default module loader. Usually one could use Quibble, ESMock, proxyquire or rewire, meaning the test above would look something like this when using Proxyquire:
// assuming that `receiver` uses `factory` internally
// comment out the import - we'll use proxyquire
// import * as factory from '../src/factory';
// import { receiver } from '../src/receiver';
const factory = { backbone_factory: sinon.stub() };
const receiver = proxyquire('../src/receiver', { './factory' : factory });
Modifying the proxyquire example to use Quibble or ESMock (both supports ESM natively) should be trivial.
Sinon needs to evolve with the times or be left behind (ESM is becoming defacto now with Node 12) as it is turning out to be a giant pain to use due to its many limitations.
This article provides a workaround (actually 4, but I only found 1 to be acceptable). In my case, I was exporting functions from a module directly and getting this error: ES Modules cannot be stubbed
export function abc() {
}
The solution was to put the functions into a class and export that instead:
export class Utils {
abc() {
}
}
notice that the function keyword is removed in the method syntax.
Happy Coding - hope Sinon makes it in the long run, but it's not looking good given its excessive rigidity.
Sticking with the questions Headline „Stub an export from a native ES Module without babel“ here's my take, using mocha and esmock:
(credits: certainly #oligofren brought me on the right path…)
package.json:
"scripts": {
...
"test": "mocha --loader=esmock",
"devDependencies": {
"esmock": "^2.1.0",
"mocha": "^10.2.0",
TestDad.js (a class)
import { sonBar } from './testSon.js'
export default class TestDad {
constructor() {
console.log(purple('constructing TestDad, calling...'))
sonBar()
}
}
testSon.js (a 'util' library)
export const sonFoo = () => {
console.log(`Original Son 'foo' and here, my brother... `)
sonBar()
}
export const sonBar = () => {
console.log(`Original Son bar`)
}
export default { sonFoo, sonBar }
esmockTest.js
import esmock from 'esmock'
describe.only(autoSuiteName(import.meta.url),
() => {
it('Test 1', async() => {
const TestDad = await esmock('../src/commands/TestDad.js', {
'../src/commands/testSon.js': {
sonBar: () => { console.log('STEPSON Bar') }
}
})
// eslint-disable-next-line no-new
new TestDad()
})
it('Test 2', async() => {
const testSon = await esmock('../src/commands/testSon.js')
testSon.sonBar = () => { console.log('ANOTHER STEPSON Bar') }
testSon.sonFoo() // still original
testSon.sonBar() // different now
})
})
autoSuiteName(import.meta.url)
regarding Test1
working nicely, import bended as desired.
regarding Test1
Bending a single function to do something else is not a problem.
(but then there is not much test value in calling your very own function you just defined, is there?)
Enclosed function calls within the module (i.e. from sonFoo to sonBar) remain what they are, they are indeed a closure, still pointing to the prior function
Btw also tested that: No better results with sinon.callsFake() (would have been surprising if there was…)
How can I detect whether my Node.JS file was called using SH:node path-to-file or JS:require('path-to-file')?
This is the Node.JS equivalent to my previous question in Perl: How can I run my Perl script only if it wasn't loaded with require?
if (require.main === module) {
console.log('called directly');
} else {
console.log('required as a module');
}
See documentation for this here: https://nodejs.org/docs/latest/api/modules.html#modules_accessing_the_main_module
There is another, slightly shorter way (not outlined in the mentioned docs).
var runningAsScript = !module.parent;
I outlined more details about how this all works under the hood in this blog post.
For those using ES Modules (and Node 10.12+), you can use import.meta.url:
import path from 'path';
import { fileURLToPath } from 'url'
const nodePath = path.resolve(process.argv[1]);
const modulePath = path.resolve(fileURLToPath(import.meta.url))
const isRunningDirectlyViaCLI = nodePath === modulePath
Things like require.main, module.parent and __dirname/__filename aren’t available in ESM.
Note: If using ESLint it may choke on this syntax, in which case you’ll need to update to ESLint ^7.2.0 and turn your ecmaVersion up to 11 (2020).
More info: process.argv, import.meta.url
I was a little confused by the terminology used in the explanation(s). So I had to do a couple quick tests.
I found that these produce the same results:
var isCLI = !module.parent;
var isCLI = require.main === module;
And for the other confused people (and to answer the question directly):
var isCLI = require.main === module;
var wasRequired = !isCLI;
Try this if you are using ES6 modules:
if (process.mainModule.filename === __filename) {
console.log('running as main module')
}
I always find myself trying to recall how to write this goddamn code snippet, so I decided to create a simple module for it. It took me a bit to make it work since accessing caller's module info is not straightforward, but it was fun to see how it could be done.
So the idea is to call a module and ask it if the caller module is the main one. We have to figure out the module of the caller function. My first approach was a variation of the accepted answer:
module.exports = function () {
return require.main === module.parent;
};
But that is not guaranteed to work. module.parent points to the module which loaded us into memory, not the one calling us. If it is the caller module that loaded this helper module into memory, we're good. But if it isn't, it won't work. So we need to try something else. My solution was to generate a stack trace and get the caller's module name from there:
module.exports = function () {
// generate a stack trace
const stack = (new Error()).stack;
// the third line refers to our caller
const stackLine = stack.split("\n")[2];
// extract the module name from that line
const callerModuleName = /\((.*):\d+:\d+\)$/.exec(stackLine)[1];
return require.main.filename === callerModuleName;
};
Save this as is-main-module.js and now you can do:
const isMainModule = require("./is-main-module");
if (isMainModule()) {
console.info("called directly");
} else {
console.info("required as a module");
}
Which is easier to remember.
First, let's define the problem better. My assumption is that what you are really looking for is whether your script owns process.argv (i.e. whether your script is responsible for processing process.argv). With this assumption in mind, the code and tests below are accurate.
module.parent works excellently, but it is deprecated for good reasons (a module might have multiple parents, in which case module.parent only represents the first parent), so use the following future-proof condition to cover all cases:
if (
typeof process === 'object' && process && process.argv
&& (
(
typeof module === 'object' && module
&& (
!module.parent
|| require.main === module
|| (process.mainModule && process.mainModule.filename === __filename)
|| (__filename === "[stdin]" && __dirname === ".")
)
)
|| (
typeof document === "object"
&& (function() {
var scripts = document.getElementsByTagName("script");
try { // in case we are in a special environment without path
var normalize = require("path").normalize;
for (var i=0,len=scripts.length|0; i < len; i=i+1|0)
if (normalize(scripts[i].src.replace(/^file:/i,"")) === __filename)
return true;
} catch(e) {}
})()
)
)
) {
// this module is top-level and invoked directly by the CLI
console.log("Invoked from CLI");
} else {
console.log("Not invoked from CLI");
}
It works correctly in all of the scripts in all of the following cases and never throws any errors†:
Requiring the script (e.x. require('./main.js'))
Directly invoking the script (e.x. nodejs cli.js)
Preloading another script (e.x. nodejs -r main.js cli.js)
Piping into node CLI (e.x. cat cli.js | nodejs)
Piping with preloading (e.x. cat cli.js | nodejs -r main.js)
In workers (e.x. new Worker('./worker.js'))
In evaled workers (e.x. new Worker('if (<test for CLI>) ...', {eval: true}))
Inside ES6 modules (e.x. nodejs --experimental-modules cli-es6.js)
Modules with preload (e.x. nodejs --experimental-modules -r main-es6.js cli-es6.js)
Piped ES6 modules (e.x. cat cli-es6.js | nodejs --experimental-modules)
Pipe+preload module (e.x. cat cli-es6.js | nodejs --experimental-modules -r main-es6.js)
In the browser (in which case, CLI is false because there is no process.argv)
In mixed browser+server environments (e.x. ElectronJS, in which case both inline scripts and all modules loaded via <script> tags are considered CLI)
The only case where is does not work is when you preload the top-level script (e.x. nodejs -r cli.js cli.js). This problem cannot be solved by piping (e.x. cat cli.js | nodejs -r cli.js) because that executes the script twice (once as a required module and once as top-level). I do not believe there is any possible fix for this because there is no way to know what the main script will be from inside a preloaded script.
† Theoretically, errors might be thrown from inside of a getter for an object (e.x. if someone were crazy enough to do Object.defineProperty(globalThis, "process", { get(){throw 0} });), however this will never happen under default circumstances for the properties used in the code snippet in any environment.
How can I detect whether my node.js file was called directly from console (windows and unix systems) or loaded using the ESM module import ( import {foo} from 'bar.js')
Such functionality is not exposed. For the moment you should separate your cli and library logic into separate files.
Answer from node.js core contributor devsnek replying to nodejs/help/issues/2420
It's the right answer in my point of view