How I can use a commonjs module in my quasar project - node.js

I an SSR Quasar project using Vite. Whenever I try to add the #tiptap/extension-code-block-lowlight extension to my project, build it and then node dist/ssr/index.js it throws the following error:
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /home/whatever/devotto/devotto.com/node_modules/lowlight/lib/common.js
require() of ES modules is not supported.
require() of /home/whatever/devotto/devotto.com/node_modules/lowlight/lib/common.js from /home/whatever/devotto/devotto.com/dist/ssr/server/server-entry.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename common.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /home/whatever/devotto/devotto.com/node_modules/lowlight/package.json.
Upon investigation, I have concluded that the issue is the lowlight library being imported by #tiptap/extension-code-block-lowlight.
If I manually go to my node_modules/#tiptap/extension-code-block-lowlight/package.json AND node_modules/lowlight/package.json and remove the line "type": "module", I can run the project with no problem (e.g. yarn build && node dist/ssr/index.js.
This solution works on my current machine but I shouldn't have to touch the node_modules folder.
I would assume that I have to transpile lowlight library which prompts me to try to alter Vite configuration but no luck there as well
module.exports = function() {
return {
build: {
extendViteConf (viteConf, { isClient, isServer }) {
if (isServer) {
viteConf.optimizeDeps = viteConf.optimizeDeps || {};
viteConf.optimizeDeps.include = ['./node_modules/highlight.js'];
viteConf.build.commonjsOptions = viteConf.build.commonjsOptions || {};
viteConf.build.commonjsOptions.include = [/highlight.js/, /node_modules/];
// viteConf.optimizeDeps.entries = [
// 'node_modules/#tiptap/extension-code-block-lowlight/dist/tiptap-extension-code-block-lowlight.cjs',
// 'node_modules/highlight.js'
// ];
}
},
}
}
}
Is there a solution to this issue without having to manually change node_module folder? Thank you very much in advance.

I didn't exactly solve the question. I only applied an automated way to handle this whenever I run the command to build the server using pre scripts.
On my package.json:
{
"scripts": {
"start:test:webserver": "ENV_FILE=test quasar build --mode ssr --port 3000 && node dist/ssr/index.js",
"prestart:test:webserver": "sed -i '/\"type\": \"module\",/d' node_modules/lowlight/package.json && sed -i '/\"type\": \"module\",/d' node_modules/#tiptap/extension-code-block-lowlight/package.json",
}
}

Related

Must use import to load ES Module - lowlight with Quasar app [duplicate]

I an SSR Quasar project using Vite. Whenever I try to add the #tiptap/extension-code-block-lowlight extension to my project, build it and then node dist/ssr/index.js it throws the following error:
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /home/whatever/devotto/devotto.com/node_modules/lowlight/lib/common.js
require() of ES modules is not supported.
require() of /home/whatever/devotto/devotto.com/node_modules/lowlight/lib/common.js from /home/whatever/devotto/devotto.com/dist/ssr/server/server-entry.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename common.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /home/whatever/devotto/devotto.com/node_modules/lowlight/package.json.
Upon investigation, I have concluded that the issue is the lowlight library being imported by #tiptap/extension-code-block-lowlight.
If I manually go to my node_modules/#tiptap/extension-code-block-lowlight/package.json AND node_modules/lowlight/package.json and remove the line "type": "module", I can run the project with no problem (e.g. yarn build && node dist/ssr/index.js.
This solution works on my current machine but I shouldn't have to touch the node_modules folder.
I would assume that I have to transpile lowlight library which prompts me to try to alter Vite configuration but no luck there as well
module.exports = function() {
return {
build: {
extendViteConf (viteConf, { isClient, isServer }) {
if (isServer) {
viteConf.optimizeDeps = viteConf.optimizeDeps || {};
viteConf.optimizeDeps.include = ['./node_modules/highlight.js'];
viteConf.build.commonjsOptions = viteConf.build.commonjsOptions || {};
viteConf.build.commonjsOptions.include = [/highlight.js/, /node_modules/];
// viteConf.optimizeDeps.entries = [
// 'node_modules/#tiptap/extension-code-block-lowlight/dist/tiptap-extension-code-block-lowlight.cjs',
// 'node_modules/highlight.js'
// ];
}
},
}
}
}
Is there a solution to this issue without having to manually change node_module folder? Thank you very much in advance.
I didn't exactly solve the question. I only applied an automated way to handle this whenever I run the command to build the server using pre scripts.
On my package.json:
{
"scripts": {
"start:test:webserver": "ENV_FILE=test quasar build --mode ssr --port 3000 && node dist/ssr/index.js",
"prestart:test:webserver": "sed -i '/\"type\": \"module\",/d' node_modules/lowlight/package.json && sed -i '/\"type\": \"module\",/d' node_modules/#tiptap/extension-code-block-lowlight/package.json",
}
}

Webpack error after upgrading Node: "Module parse failed: Unexpected token"

I'm troubleshooting a webpack error.
Command: bin/webpack --colors --progress
Produces this error:
ERROR in ./node_modules/#flatfile/sdk/dist/index.js 351:361
Module parse failed: Unexpected token (351:361)
File was processed with these loaders:
* ./node_modules/babel-loader/lib/index.js
You may need an additional loader to handle the result of these loaders.
| class v extends i {
| constructor(e, t) {
> super(e), r(this, "code", "FF-UA-00"), r(this, "name", "UnauthorizedError"), r(this, "debug", "The JWT was not signed with a recognized private key or you did not provide the necessary information to identify the end user"), r(this, "userMessage", "There was an issue establishing a secure import session."), e && (this.debug = e), this.code = t ?? "FF-UA-00";
| }
| }
# ./app/javascript/src/app/pages/content_assets/Index.vue?vue&type=script&lang=ts& (./node_modules/babel-loader/lib??ref--8-0!./node_modules/vue-loader/lib??vue-loader-options!./app/javascript/src/app/pages/content_assets/Index.vue?vue&type=script&lang=ts&) 22:0-41 125:6-14
# ./app/javascript/src/app/pages/content_assets/Index.vue?vue&type=script&lang=ts&
# ./app/javascript/src/app/pages/content_assets/Index.vue
# ./app/javascript/packs/app.js
NOTES
I found what appears to be an identical issue reported in the Flatfile project: https://github.com/FlatFilers/sdk/issues/83
Looks like ES2020 was emitted to the /dist folder so my cra babel loader is not able to parse it, in order to fix it I need to include the path on my webpack config.
Node v16.13.1
We're using webpack with a Rails project via the webpacker package (#rails/webpacker": "5.4.3") which is depending on webpack#4.46.0.
When I change to Node v14x and rebuild node_modules (yarn install) webpack compiles successfully.
The line referenced in the error (351:361) does not exist when I go check the file in node_modules/
We have a yarn.lock file, which I delete and recreate before running yarn install. I also delete the node_modules directory to ensure a "fresh" download of the correct packages.
We have a babel.config.js file...
module.exports = function(api) {
var validEnv = ['development', 'test', 'production']
var currentEnv = api.env()
var isDevelopmentEnv = api.env('development')
var isProductionEnv = api.env('production')
var isTestEnv = api.env('test')
if (!validEnv.includes(currentEnv)) {
throw new Error(
'Please specify a valid `NODE_ENV` or ' +
'`BABEL_ENV` environment variables. Valid values are "development", ' +
'"test", and "production". Instead, received: ' +
JSON.stringify(currentEnv) +
'.'
)
}
return {
presets: [
isTestEnv && [
'#babel/preset-env',
{
targets: {
node: 'current'
}
}
],
(isProductionEnv || isDevelopmentEnv) && [
'#babel/preset-env',
{
forceAllTransforms: true,
useBuiltIns: 'entry',
corejs: 3,
modules: false,
exclude: ['transform-typeof-symbol']
}
],
["babel-preset-typescript-vue", { "allExtensions": true, "isTSX": true }]
].filter(Boolean),
plugins: [
'babel-plugin-macros',
'#babel/plugin-syntax-dynamic-import',
isTestEnv && 'babel-plugin-dynamic-import-node',
'#babel/plugin-transform-destructuring',
[
'#babel/plugin-proposal-class-properties',
{
loose: true
}
],
[
'#babel/plugin-proposal-object-rest-spread',
{
useBuiltIns: true
}
],
[
'#babel/plugin-transform-runtime',
{
helpers: false,
regenerator: true,
corejs: false
}
],
[
'#babel/plugin-transform-regenerator',
{
async: false
}
]
].filter(Boolean)
}
}
Ultimately I want to get webpack to compile. If you had advice about any of the following questions, it would help a lot.
Why would changing the Node version (only) cause different webpack behavior? We aren't changing the the webpack version or the version of the #flatfile package that's causing the error.
Why is the error pointing to a line that doesn't exist in the package? Is this evidence of some kind of caching problem?
Does the workaround mentioned in the linked GitHub issue shed light on my problem?
I'll take a stab at this.
I believe your issue is that webpack 4 does not support the nullish coalescing operator due to it's dependency on acorn 6. See this webpack issue and this PR comment.
You haven't specified the exact minor version of Node.js 14x that worked for you. I will assume it was a version that did not fully support the nullish coalescing operator, or at least a version that #babel/preset-env's target option understood to not support ??, so it was transpiled by babel and thus webpack didn't complain. You can see what versions of node support nullish coalescing on node.green.
I don't fully understand the point you are making here, so not focusing on this in the proposed solution.
I'm not sure what the proposed workaround is in the linked issue, maybe the comment about "include the path on my webpack config", but yes the issue does seem relevant as it is pointing out the nullish coalescing operator as the source of the issue.
You can try to solve this by
adding #babel/plugin-proposal-nullish-coalescing-operator to your babel config's plugins
updating your webpack config to run #flatfile/sdk through babel-loader to transpile the nullish coalescing operator:
{
test: /\.jsx?$/,
exclude: filename => {
return /node_modules/.test(filename) && !/#flatfile\/sdk\/dist/.test(filename)
},
use: ['babel-loader']
}
Another possibility is to upgrade webpacker to a version that depends upon webpack v5.
One final remark, when you say
We have a yarn.lock file, which I delete and recreate before running yarn install.
you probably should not be deleting the lock file before each install, as it negates the purpose of a lock file altogether.

ERR_REQUIRE_ESM from webpack's importing an ESM package

I am building an app using Nx for modularisation and Geckos for the server part. With Nx' default Webpack 5 config for Node I get this error when running the app:
/Users/hgrzeskowiak/hugo-game/fresh-nx-geckos/nx-geckos-test/dist/apps/geckos-server-app/main.js:8
module.exports = require("#geckos.io/server");
^
Error [ERR_REQUIRE_ESM]: require() of ES Module /Users/hgrzeskowiak/hugo-game/fresh-nx-geckos/nx-geckos-test/node_modules/#geckos.io/server/lib/index.js from /Users/hgrzeskowiak/hugo-game/fresh-nx-geckos/nx-geckos-test/dist/apps/geckos-server-app/main.js not supported.
Instead change the require of index.js in /Users/hgrzeskowiak/hugo-game/fresh-nx-geckos/nx-geckos-test/dist/apps/geckos-server-app/main.js to a dynamic import() which is available in all CommonJS modules.
(...)
Here are steps to reproduce:
$ npx create-nx-workspace#latest nx-geckos-test
Need to install the following packages:
create-nx-workspace#latest
Ok to proceed? (y) y
✔ What to create in the new workspace · express
✔ Application name · geckos-server-app
✔ Use Nx Cloud? (It's free and doesn't require registration.) · No
> NX Nx is creating your v13.4.1 workspace.
To make sure the command works reliably in all environments, and that the preset is applied correctly,
Nx will run "npm install" several times. Please wait.
✔ Installing dependencies with npm
✔ Nx has successfully created the workspace.
$ npm run start
> nx-geckos-test#0.0.0 start
> nx serve
> nx run geckos-server-app:serve
chunk (runtime: main) main.js (main) 552 bytes [entry] [rendered]
webpack compiled successfully (26ac6f288b082854)
Debugger listening on ws://localhost:9229/70a1d9b4-af2b-4fc9-90f7-f3502e5fdf2e
Debugger listening on ws://localhost:9229/70a1d9b4-af2b-4fc9-90f7-f3502e5fdf2e
For help, see: https://nodejs.org/en/docs/inspector
Issues checking in progress...
Listening at http://localhost:3333/api
At this stage, the default Express app builds and runs.
Now I replace the contents of apps/geckos-server-app/src/main.ts to the Geckos server example from the README:
import geckos from '#geckos.io/server'
const io = geckos()
io.listen(3000) // default port is 9208
io.onConnection(channel => {
channel.onDisconnect(() => {
console.log(`${channel.id} got disconnected`)
})
channel.on('chat message', data => {
console.log(`got ${data} from "chat message"`)
// emit the "chat message" data to all channels in the same room
io.room(channel.roomId).emit('chat message', data)
})
})
...and install the geckos server package using
$ npm install #geckos.io/server
added 4 packages, and audited 785 packages in 7s
87 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
Ok, time to run it again:
$ npm run start
> nx-geckos-test#0.0.0 start
> nx serve
> nx run geckos-server-app:serve
chunk (runtime: main) main.js (main) 610 bytes [entry] [rendered]
webpack compiled successfully (90920432a9d246ea)
Debugger listening on ws://localhost:9229/dc6d9e1a-1a15-432f-a46c-d2ac83b923ea
Debugger listening on ws://localhost:9229/dc6d9e1a-1a15-432f-a46c-d2ac83b923ea
For help, see: https://nodejs.org/en/docs/inspector
/Users/hgrzeskowiak/hugo-game/fresh-nx-geckos/nx-geckos-test/dist/apps/geckos-server-app/main.js:8
module.exports = require("#geckos.io/server");
^
Error [ERR_REQUIRE_ESM]: require() of ES Module /Users/hgrzeskowiak/hugo-game/fresh-nx-geckos/nx-geckos-test/node_modules/#geckos.io/server/lib/index.js from /Users/hgrzeskowiak/hugo-game/fresh-nx-geckos/nx-geckos-test/dist/apps/geckos-server-app/main.js not supported.
Instead change the require of index.js in /Users/hgrzeskowiak/hugo-game/fresh-nx-geckos/nx-geckos-test/dist/apps/geckos-server-app/main.js to a dynamic import() which is available in all CommonJS modules.
at Object.#geckos.io/server (/Users/hgrzeskowiak/hugo-game/fresh-nx-geckos/nx-geckos-test/dist/apps/geckos-server-app/main.js:8:18)
at __webpack_require__ (/Users/hgrzeskowiak/hugo-game/fresh-nx-geckos/nx-geckos-test/dist/apps/geckos-server-app/main.js:32:41)
at /Users/hgrzeskowiak/hugo-game/fresh-nx-geckos/nx-geckos-test/dist/apps/geckos-server-app/main.js:45:18
at /Users/hgrzeskowiak/hugo-game/fresh-nx-geckos/nx-geckos-test/dist/apps/geckos-server-app/main.js:59:3
at Object.<anonymous> (/Users/hgrzeskowiak/hugo-game/fresh-nx-geckos/nx-geckos-test/dist/apps/geckos-server-app/main.js:64:12)
I tested quite a few Webpack config overrides and using type: "module" in package.json, but to no avail. The output main.js looks mostly the same no matter what webpack settings I change. Things I tried changing in the app's webpack override config:
module.exports = function(webpackConfig, context) {
webpackConfig.target = "async-node16";
webpackConfig.output.libraryTarget = "module";
webpackConfig.output.library = {type: "module"};
webpackConfig.output.chunkFormat = "module";
webpackConfig.experiments.outputModule = true;
console.log("webpack config:");
console.log(webpackConfig);
console.log("module rules:");
console.log(webpackConfig.module.rules);
return webpackConfig;
}
The console logs in the config verify that the script is being run.
The app works when I manually change the require to import in the transpiled output, but that's not a sustainable solution.
I suspect the error has something to do with Geckos being an ESM package, and Webpack only using require instead of import as it should, but I couldn't find any way of changing that behaviour.
Any ideas?
Node version 16.13.0
Nx version 13.4.1
Webpack (transitive dep of Nx): 5.65.0
Here's the cleanest solution I could find that works with Type Script versions older than 4.6 (which should add support for this feature). This is essentially patching ESM support into WebPack with older TypeScript versions (including latest at the time of writing).
A custom Webpack plugin that changes the type of the external module to import.
// TypeScript before version 4.6 does not support transpiling ESM imports to
// `import()`, but uses `require()` instead. NodeJS does not support the use
// of `require() for ECMAScript modules (ESM).
//
// Good reads:
// https://www.typescriptlang.org/docs/handbook/esm-node.html
// https://devblogs.microsoft.com/typescript/announcing-typescript-4-5/#esm-nodejs
/**
* A Webpack 5 plugin that can be passed a list of packages that are of type
* ESM. The typescript compiler will then be instructed to use the `import`
* external type.
*/
class ESMLoader {
static defaultOptions = {
esmPackages: "all"
};
constructor(options = {}) {
this.options = { ...ESMLoader.defaultOptions, ...options };
}
apply(compiler) {
compiler.hooks.compilation.tap(
"ECMAScript Module (ESM) Loader. Turns require() into import()",
(
compilation
) => {
compilation.hooks.buildModule.tap("Hello World Plugin", (module) => {
if (
module.type === "javascript/dynamic" &&
(
this.options.esmPackages === "all" ||
this.options.esmPackages.includes(module.request)
)
) {
// All types documented at
// https://webpack.js.org/configuration/externals/#externalstype
module.externalType = "import";
}
}
)
;
}
);
}
}
module.exports = function(webpackConfig, context) {
// NodeJS supports dynamic imports since version 12.
webpackConfig.target = "node16";
webpackConfig.plugins.push(
// Manually specify the ESM modules or leave blank for using `import()`
// on all packages.
//new ESMLoader()
new ESMLoader({esmPackages: "#geckos.io/server"})
);
return webpackConfig;
};
The custom webpack config can be set on an Nx app through the project.json:
{
"targets": {
"build": {
"executor": "#nrwl/node:build",
"options": {
...
"webpackConfig": "apps/your-app/webpackConfig.js"
},
...
This solution also requires tsconfig.json to have set the module type to something more modern than commonjs, e.g. es2020:
{
"extends": "./tsconfig.json",
"compilerOptions": {
"outDir": "../../dist/out-tsc",
"module": "es2020",
"types": ["node"],
},
"exclude": ["**/*.spec.ts", "**/*.test.ts"],
"include": ["**/*.ts"]
}
Once this is done, the built file can be ran using the usual nx serve.

npm package.json aliases like webpack

i am trying to alias a module however i am not sure how to do that with package.json
in webpack you would do something like this:
module.exports = {
//...
resolve: {
alias: {
'pixi.js': 'pixi.js-legacy'
}
}
};
But what is the equivalent without webpack?
Since NPM Version 6.9 of March 2019 it is supported without installing any additional packages (see the RFC):
npm i aliasName#npm:packageToInstall
⬇⬇⬇
// package.json
"dependencies": {
"aliasName": "npm:packageToInstall#^1.6.1"
}
The idea seems to be that npm: is a URI-like scheme in a dependency version specifier.
Usage:
const alias = require( 'aliasName' );
There is a npm package for this: module-alias.
After installing it you can add your aliases to the package.json, like so:
"_moduleAliases": {
"#root" : ".", // Application's root
"#deep" : "src/some/very/deep/directory/or/file",
"#my_module" : "lib/some-file.js",
"something" : "src/foo", // Or without #. Actually, it could be any string
}
Make sure to add this line at the top of your app's main file:
require('module-alias/register');
You should only use this in final products (and not packages you intend to publish in npm or use elsewhere) - it modifies the behavior of require.

Importing typescript from external node modules

I want to split my application into different node modules and have a main module which builds all other modules as well and I want to use typescript with es6 modules.
Here is my planned project structure:
main
node_modules
dep-a
dep-b
framework
interfaces
IComponent.ts
dep-a
components
test.ts
node_modules
framework
index.ts
dep-b
node_modules
framework
I want to be able to define interfaces in framework which can be consumed in dep-a, dep-b and main.
How do I set up this correctly? Can I compile everything from my main-module? Do I need to create different bundles for framework, dep-a, ... and another typing file? What is the best approach for this?
I already set up some test files and folders and used npm link to link the dependencies and webpack to bundle the files and I am always running into issues with files not being found:
error TS2307: Cannot find module 'framework/interfaces/IComponent'
and
Module not found: Error: Cannot resolve 'file' or 'directory' ./components/test
TL;DR generate declarations for the modules using declaration: true in tsconfig.json and specify the file for your generated typings in the typings entry of the package.json file
framework
Use a tsconfig file similar to this:
{
"compilerOptions": {
"target": "es5",
"module": "commonjs",
"declaration": true,
"noImplicitAny": true,
"removeComments": true,
"outDir": "dist",
...
},
"files": [
...
]
}
The important bit is declaration: true which will generate internal declarations in the dist directory
Assuming there is an index.ts file which (re)exports all the interesting parts of framework, create a package.json file with a main and typings entry pointing to, respectively, the generated js and the generated declaration, i.e.
{
"name": "framework",
"main": "dist/index.js",
"typings": "dist/index.d.ts",
...
}
Commit this module to a git repo, say bitbucket at : "https://myUser#bitbucket.org/myUser/framework.git"
dep-a
in package.json create a dependency to framework
{
"dependencies": {
"framework": "https://myUser#bitbucket.org/myUser/framework.git"
},
}
That is it.
import * from 'framework'
will pull the dependency with the typings, automatically
Obviously, it is possible to do with dep-a what was done with framework i.e. generate the declarations, update package.json and use dep-a as a module with embedded typings in main
note: a file URL will do in package.json/dependencies if you do not want go to via an external git repo
What arrived in TypeScript 1.6 is typings property in package.json module. You can check the relevant issue on GitHub.
So assuming you want to create separate modules ( dep-a, framework ). You can do the following :
main.ts // (1)
package.json // (2)
node_modules/
dep_a/
index.js // (3)
index.d.ts // (4)
package.json // (5)
node_modules/
framework/
index.js // (6)
index.d.ts // (7)
package.json // (8)
So let's see what you have in your files :
//(1) main.ts
import * as depA from "depA";
console.log(depA({ a : true, b : 2 }) === true) // true;
//(2) package.json
{
name: "main",
dependencies: {
"dep_a" : "0.0.1"
}
...
}
For depA
//(3) dep_a/index.js
module.exports = function a(options) { return true; };
//(4) dep_a/index.d.ts;
import * as framework from "framework";
export interface IDepA extends framework.IFramework {
a : boolean
}
export default function a(options: IDepA) : boolean;
//(5) dep_a/package.json
{
name: "dep_a",
dependencies: {
"framework" : "0.0.1"
},
...
typings : "index.d.ts" // < Magic happens here
}
For framework
//(6) dep_a/node_modules/framework/index.js
module.exports = true // we need index.js here, but we will only use definition file
//(7) dep_a/node_modules/framework/index.d.ts;
export interface IFramework {
b : number;
}
//(8) dep_a/node_modules/framework/package.json
{
name: "framework"
...
typings : "index.d.ts"
}
What I don't include in this answer ( for clarity ) is another compilation phase, so you could actually write the modules ( dep_a, framework ) with typescript and then compile them to index.js before you use them.
For a detailed explanation and some background also see: https://medium.com/#mweststrate/how-to-create-strongly-typed-npm-modules-1e1bda23a7f4

Resources