Webpack 5 webpackMissingModule error for node_modules package - node.js

I've been working on upgrading a repository from Webpack version 4 to 5, and I've encountered a very strange problem where Webpack will throw an error at runtime telling me that a package in node_modules could not include a specific "module" (referring to a subdirectory of that package). The module in question is #hapi/joi, and the error I'm receiving is:
Error: Cannot find module './types/alternatives'
At one point it was telling me that the error related to that package, but now it just throws the error with no package context. I've dug into the #hapi/joi package, and I can see this set of imports in the index.js:
const internals = {
alternatives: require('./types/alternatives'),
array: require('./types/array'),
boolean: require('./types/boolean'),
binary: require('./types/binary'),
date: require('./types/date'),
func: require('./types/func'),
number: require('./types/number'),
object: require('./types/object'),
string: require('./types/string'),
symbol: require('./types/symbol')
};
I've verified that the ./types/alternatives directory exists and that it has an index.js of its own. This package also works fine on Webpack 4. I tried upgrading to the standalone joi package, and it threw the same error.
I'm pretty confused at to why this error is throw at runtime. I would think this would be a build error, though maybe it has something to do with building the target for node?
This is my Webpack config. It's mostly unmodified from v4:
{
entry: {
adviceServer: "./src/server/index.ts",
},
target: "node",
bail: true,
mode: env.NODE_ENV,
output: {
path: path.resolve(root, "dist", "js"),
filename: `[name].${env.ASSET_HASH}.js`
},
module: {
rules: [
{
test: /\.tsx?$/,
use: [
{
loader: "ts-loader",
options: {
context: path.resolve(root, "src"),
configFile: tsconfigPath,
transpileOnly: false
}
}
],
exclude: [
"/node_modules",
"/**/*.test.ts/"
]
},
{
test: /\.s?css$/,
use: ["isomorphic-style-loader", "css-loader", "sass-loader"]
}
]
},
resolve: {
extensions: [".tsx", ".ts", ".js", ".jsx", "scss", ".json"],
alias: {
styles: path.resolve(root, "src", "scss"),
canvas: path.resolve(root, "src", "etc", "fileMock"),
ws: path.resolve(root, "src", "etc", "fileMock")
},
plugins: [
new TsconfigPathsPlugin({
configFile: tsconfigPath
})
]
},
devtool: "eval-cheap-source-map",
plugins: [
new BundleTracker({ filename: "./webpack-stats.json" })
]
}

I had a similar issue upgrading from webpack 4 but with a different external library. In my case it was caused by two webpack#5 versions installed at the same time. Check your package-lock.json/yarn.lock and add overrides/resolutions to your package.json if there are several versions. If not — try removing your node_modules folder and installing packages again.

Related

NodeJs Lambda Layers and webpack (without serverless)

I've got a lambda function which is connecting to some layers. It's built using nodejs/typescript and I'm trying to get it built using webpack.
My problem is that I can't seem to figure out how to package the app using webpack with my layer module imports.
For example, I have a reference like this in my application:
import { ProductEntity, IProduct, productEntityManager } from "/opt/nodejs/orm";
If I try to run webpack normally, i'll get a Module not found error saying it can't find "/opt/nodejs/orm".
So I've added in the ignore plugin as below:
"use strict";
const path = require("path");
const webpack = require('webpack')
const ignore = new webpack.IgnorePlugin({resourceRegExp:/^(\/opt\/nodejs\/search|\/opt\/nodejs\/orm|\/opt\/nodejs\/put-event)$/})
module.exports = {
devtool: "source-map",
entry: "./src/handler.ts",
mode: "production",
target: "node",
plugins: [ignore],
node: {
__dirname: true,
},
output: {
filename: "index.js",
libraryTarget: "commonjs2",
path: path.resolve(__dirname, ".build"),
},
module: {
rules: [
{
test: /\.(graphql|gql)$/,
loader: "graphql-tag/loader",
exclude: /node_modules/,
},
{
test: /\.(tsx?)$/,
loader: "ts-loader",
exclude: [
[
path.resolve(__dirname, "node_modules"),
path.resolve(__dirname, ".serverless"),
path.resolve(__dirname, ".webpack")
],
],
options: {
transpileOnly: false,
experimentalWatchApi: true,
},
},
],
},
resolve: {
extensions: [".ts", ".tsx", ".js"],
},
};
And now I get the same error, but it's just baked in to my packaged js file.
I get something like this appear in that bundled js file:
...
var e=new Error("Cannot find module '/opt/nodejs/orm'")
...
So my question is... How do I build my lambda function using webpack and get it to not try and resolve or import the lambda layer modules?
I can see a lot of examples using serverless but I'm not using it (and not planning to because I'm using terraform).

Webpack bundling but not resolving typescript from node_modules package

Question guys, I've tried about every different way I can think of to solve this.
We have two folders in a makeshift mono-repo (no yarn workspace). One called Mgt-Shared one called Server. We reference shared from Servers package.json with "mgt-shared": "file:../mgt-shared", and I run a simple npm install to get our shared into Servers node_modules.
Both projects are mixed typescript and javascript.
When I run webpack to attempt to bundle server, it works fine, the Typescript files from shared show up in the bundle list but when I run the node dist/app.bundle.js - it can't resolve some of the modules (specifically all the typescript ones).
When I run webpack --json I find this (which seems like it's resolving properly):
{
"id": "mgt-shared/discounts/discount_reasons",
"identifier": "external \"mgt-shared/discounts/discount_reasons\"",
"name": "external \"mgt-shared/discounts/discount_reasons\"",
"index": 23,
"index2": 15,
"size": 42,
"built": true,
"optional": false,
"prefetched": false,
"chunks": [
"main"
],
...
"failed": false,
"errors": 0,
"warnings": 0,
"assets": [],
"reasons": [
{
"moduleId": "./src/discounts/discount.ts",
"moduleIdentifier": "/home/mygastank/WebstormProjects/mygastank/server/node_modules/babel-loader/lib/index.js!/home/mygastank/WebstormProjects/mygastank/server/src/discounts/discount.ts",
"module": "./src/discounts/discount.ts",
"moduleName": "./src/discounts/discount.ts",
"type": "cjs require",
"userRequest": "mgt-shared/discounts/discount_reasons",
"loc": "20:24-72"
},
{...
],
"providedExports": null,
"optimizationBailout": [],
"depth": 6
},
My webpack config is this:
const nodeExternals = require('webpack-node-externals')
module.exports = {
mode: 'development',
target: 'node',
// This forces resolution of native node modules
externals: [nodeExternals({ modulesFromFile: true })],
entry: './src/index',
output: {
filename: 'app.bundle.js'
},
resolve: {
extensions: ['.ts', '.tsx', '.js', '.json'],
},
module: {
rules: [
{
// Include ts, tsx, js, and jsx files.
test: /\.(ts|js)?$/,
loader: 'babel-loader',
include: [/node_modules\/mgt-shared/],
exclude: [/node_modules/],
options: {
presets: [
[
'env',
{
'targets': {
'node': 'current'
}
}
],
['#babel/preset-typescript',
{ 'allExtensions': true, 'isTSX': true }]
]
}
}]
}
}
I found that there was a way to whitelist other directories.
I needed to add the whitelist key to the options for nodeExternals like so:
externals: [nodeExternals({
whitelist: [/mgt-shared/],
modulesFromFile: true
})],
Everything else was able to stay the same.

How to include only specific module in webpack without transpiling its dependencies?

I'm working with a monorepo node.js project with the following structure:
rootDir
packageA
packageB
packageC
I want to produce a bundle of packageB code using webpack and babel.js. packageA has a lot of exports however only one export is used by packageA, let's call the export utils.
When I run webpack it recognizes that packageB depends on packageA so webpack dependecnies graph now includes all modules exported by packageA in its index.js although only one is really needed. So all of the modules in packageA are being transpiled by babel.js which even includes the node_modules of packages included by files in packageA. I've run into numerous issues with transpiling node_modules with babel.js. They range from babel.js not being able to deal with non-Javascript files for which loaders do not exist, webpack not finding certain modules, having mixed esm and commonjs syntax inside some files and many others*.
Other issues could only be solved by commenting out 2 problematic modules exported by packageA.
As you can see the solution is not ideal and perhaps not sustainable for long. Is there a way to have webpack transpile packageA without transpiling its node_modules? I couldn't achieve this despite my numerous regex attempts. Below are my config files:
*I managed to solve the issue by adding modules: "commonjs" in babel config however this will disable webpack tree shaking mechanism which means that I need to use a separate babelrc for my project.
webpack.config.js
const path = require('path')
const webpack = require('webpack')
module.exports = {
mode: 'development',
target: 'node',
entry: './src/index.js',
output: {
filename: 'bundle.js',
path: path.resolve(__dirname, 'build')
},
devtool: 'source-map',
plugins: [
new webpack.IgnorePlugin(/^(hiredis|transifex)$/)
],
module: {
rules: [
{
test: /node_modules\/datauri\/index\.js$/,
loaders: ['shebang-loader', 'babel-loader']
},
{
test: /node_modules\/needle\/lib\/needle\.js$/,
loaders: ['file-loader', 'babel-loader']
},
{
test: /\.js?$/,
use: {
loader: 'babel-loader',
options: {
rootMode: 'upward'
}
},
include: [
path.resolve(__dirname, 'src'),
"/path/to/packageA",
"/path/to/packageC",
]
}
]
}
}
babel.config.js
module.exports = {
plugins: [
[
'#babel/plugin-proposal-decorators',
{
legacy: true
}
],
'#babel/plugin-syntax-dynamic-import',
'#babel/plugin-syntax-import-meta',
'#babel/plugin-proposal-class-properties',
'#babel/plugin-proposal-json-strings',
'#babel/plugin-proposal-function-sent',
'#babel/plugin-proposal-export-namespace-from',
'#babel/plugin-proposal-numeric-separator',
'#babel/plugin-proposal-throw-expressions',
'#babel/plugin-proposal-export-default-from',
'#babel/plugin-proposal-logical-assignment-operators',
'#babel/plugin-proposal-optional-chaining',
[
'#babel/plugin-proposal-pipeline-operator',
{
proposal: 'minimal'
}
],
'#babel/plugin-proposal-nullish-coalescing-operator',
'#babel/plugin-proposal-do-expressions',
'#babel/plugin-proposal-function-bind'
],
presets: [
[
'#babel/preset-env',
{
targets: {
node: 'current'
},
corejs: 3,
useBuiltIns: 'usage',
modules: "commonjs"
}
],
'#babel/preset-typescript'
],
env: {
debug: {
sourceMaps: 'inline',
retainLines: true
}
}
}
UPDATE: I just noticed that I'm not ignoring node_modules in babel.config.js so I added:
ignore: [
/node_modules/
]
to my babel.config.js. This allowed me to stop using modules: 'commonjs' so I guess this did have some effect but I still get errors when trying to run the bundle, errors that resulted from transpiling node_modules for example this error.

Unable to load sp-loader for spfx solution with webpack

my typescript file includes the following import:
import { SPComponentLoader } from '#microsoft/sp-loader';
But I get a lot of errors when building with webpack
npx webpack --config webpack.config.js
Here are some of the errors:
ERROR in
./node_modules/#microsoft/sp-loader/lib/requirejs/RequireJsLoader.js
Module not found: Error: Can't resolve './test/RequireJsMock' in
'C:\users\agaskell\source\repos\spfxBanner\node_modules#microsoft\sp-loader\lib\requirejs'
# ./node_modules/#microsoft/sp-loader/lib/requirejs/RequireJsLoader.js
258:14-45 #
./node_modules/#microsoft/sp-loader/lib/requirejs/SPRequireJsComponentLoader.js
# ./node_modules/#microsoft/sp-loader/lib/starter/SPStarter.js #
./node_modules/#microsoft/sp-loader/lib/index.js #
./Classic/client/bootHeader.ts # multi #babel/polyfill
./Classic/client/bootHeader.ts
ERROR in
./node_modules/#microsoft/sp-loader/lib/systemjs/SystemJsLoader.js
Module not found: Error: Can't resolve './test/SystemJsMock' in
'C:\users\agaskell\source\repos\spfxBanner\node_modules#microsoft\sp-loader\lib\systemjs'
I am trying to build my ts file into js for classic SharePoint sites and I normally use gulp for modern pages, but for classic I am using a separate bootloader.ts file and webpack.
Can anyone help?
Here is the webpack.config.js file:
const path = require("path");
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
module.exports = {
mode: "development",
entry: ['#babel/polyfill',
path.resolve(__dirname, './Classic/client/bootHeader.ts')],
module: {
rules: [
{
test: /\.tsx?$/,
use: "ts-loader",
exclude: /node_modules/
},
{
test: /\.(s*)css$/,
use: [
// fallback to style-loader in development
process.env.NODE_ENV !== "production"
? "style-loader"
: MiniCssExtractPlugin.loader,
"css-loader",
"sass-loader"
]
},
{
test: /\.(png|jp(e*)g|svg)$/,
use: [
{
loader: "url-loader",
options: {
limit: 15000, // Convert images < 8kb to base64 strings
name: "images/[hash]-[name].[ext]"
}
}
]
}
]
},
plugins: [
new MiniCssExtractPlugin({
filename: "[name].css",
chunkFilename: "[id].css"
})
],
resolve: {
extensions: [".tsx", ".ts", ".js"]
},
output: {
filename: "classicBundleAG.js",
path: path.resolve(__dirname, "Classic"),
libraryTarget: "umd"
}
};
I ended up using a workaround for this. I gave up on SPComponentLoader to load my bootstrap and instead installed bootstrap modules locally and then referenced them from my custom sass.
My thoughts are that gulp with yeoman normally handles the SPComponentLoader dependencies, but this time I am using a custom webpack and I did not want to deal with every missing dependency manually.

Bundle sha3/binary modules with Webpack

Webpack warns when I bundle my source as it can't resolve the 'sha3' module.
$ npm run build
WARNING in ./~/keccakjs/index.js
Module not found: Error: Can't resolve 'sha3' in '<PROJ>\node_modules\keccakjs'
# ./~/keccakjs/index.js 2:19-34
# ./~/<lib>/index.js
# ./lib/<file>.js
Reason being the sha3 library has no js files.
Creating library <proj>\node_modules\sha3\build\Release\sha3.lib and object <proj>\node_modules\sha3\build\Release\sha3.exp
I am able to run require('sha3') in my project, however webpack cannot resolve it.
I looked at the docs here, regarding how webpack resolves libs.
Could someone point me to how I can include sha3 in/with my bundle.
My Webpack config:
module.exports = {
target: 'node',
entry: "./<lib>.js",
devtool: "source-map",
node: {
__dirname: false,
__filename: false,
},
output: {
path: "./dist",
filename: "<lib>.min.js"
},
plugins: [
new webpack.optimize.OccurrenceOrderPlugin(),
new webpack.DefinePlugin({
'process.env.NODE_ENV': JSON.stringify('production')
})
]
}
What actually ended up working for me was:
resolve: {
alias: {
sha3: path.join(__dirname,'node_modules/sha3/build/Release/sha3.node')
},
},
module: {
rules: [
{test: /\.node$/, use: 'node-loader'},
]
},
That way I told it which file to import, when it wasn't able to resolve sha3. And the node-loader packages in the .node file!
Try using the binary loader from webpack from here. Then you can:
1) Define loaders in your WebPack config:
module.exports = {
target: 'node',
entry: "./<lib>.js",
devtool: "source-map",
node: {
__dirname: false,
__filename: false,
},
output: {
path: "./dist",
filename: "<lib>.min.js"
},
plugins: [
new webpack.optimize.OccurrenceOrderPlugin(),
new webpack.DefinePlugin({
'process.env.NODE_ENV': JSON.stringify('production')
})
],
module: {
loaders: [
{ test: /sha3$/, loader: 'binary' }
]
}
}
2) Use the loader directly in your import:
require('binary!sha3');

Resources