FS - Use FS for Electron on Webpack 5 - node.js

I recently upgraded my Quasar framework to v2 which uses Vue3, Electron 16 and Webpack 5.
When I tried to run my application, I keep getting this error:
Module not found: Can't resolve imported dependency "fs"
I understand that webpack 5 does not automatically do polyfills so I added these in my quasar.conf.js:
chainWebpack (chain) {
const nodePolyfillWebpackPlugin = require('node-polyfill-webpack-plugin')
chain.plugin('node-polyfill').use(nodePolyfillWebpackPlugin)
}
But this doesn't support polyfills for the fs module.
When I tried adding this:
if (!cfg.resolve.fallback) {
cfg.resolve.fallback = {}
cfg.resolve.fallback.fs = false
}
Now I get an error:
TypeError: fs.readFileSync is not a function
I also tried adding this in my package.json, same behavior (as expected):
"browser": {
"fs": false
}
But then.. How can I use fs in my electron application?

Was able to figure it out now.
Adding it here in case anyone might need it in the future. I see a lot of people asking this question but got no resolution.
So.. I added this to electron-preload.js, since fs / built-in node modules are accessible via the renderer:
import { contextBridge } from 'electron'
const fs = require('fs')
contextBridge.exposeInMainWorld('electronFs', {
readFileSync: fs.readFileSync,
existsSync: fs.existsSync
// Other fs methods here
})
And use it like this in Vue / JS / modules:
const fs = window.electronFs
const jsonContent = JSON.parse(fs.readFileSync(SOME_PATH))

You can solve Can't resolve imported dependency "fs" error by adding the code below to "build" section in quasar.conf.js:
extendWebpack(cfg) {
cfg.resolve.fallback = {
fs: false
}
}
quasar.conf.js:
module.exports = configure(function (ctx) {
return {
build: {
extendWebpack(cfg) {
cfg.resolve.fallback = {
fs: false
}
}
}
}
});
In addition, if there are more Can't resolve imported dependency "crypto", "Buffer", "os", "path", "stream", "assert" and so on, you can add them as shown below:
quasar.conf.js:
module.exports = configure(function (ctx) {
return {
build: {
extendWebpack(cfg) {
cfg.resolve.fallback = {
fs: false,
crypto: false,
Buffer: false,
os: false,
path: false,
stream: false,
assert: false,
dns: false,
net: false,
tls: false,
http2: false,
https: false,
http: false,
zlib: false
}
}
}
}
});
"node-polyfill-webpack-plugin" cannot cover all Can't resolve imported dependency "xxx" errors and if you fully use my solution above, you don't need to use "node-polyfill-webpack-plugin". In other words, you can use "node-polyfill-webpack-plugin" to cover some parts of Can't resolve imported dependency "xxx" errors then for the rest of Can't resolve imported dependency "xxx" errors, you use my solution above so you can use 2 solutions at the same time.

There is a new flag in Webpack 5 to target the build for Electron, which includes the correct support.
Simply add target = 'electron-main' to your webpack config.
See more here: https://webpack.js.org/configuration/target/

Related

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default

I'm trying to use Socket.io in my Laravel/Vue.js app. But I'm getting this error when running npm run dev.
Module not found: Error: Can't resolve 'path' in '/home/nicolas/LaravelProjects/Chess/node_modules/socket.io/dist'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "path": require.resolve("path-browserify") }'
- install 'path-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "path": false }
webpack compiled with 9 errors
I tried changing the webpack.config.js file by adding resolve.fallback: { "path": false }, but it doesn't seem to help. It is possible that I'm doing it wrong since I have 4 webpack.config.js files in my project (I don't know why).
Maybe can I downgrade webpack? Thanks!
This fix worked for me (Vue 3):
In vue.config.js, add:
const { defineConfig } = require('#vue/cli-service')
const NodePolyfillPlugin = require("node-polyfill-webpack-plugin")
module.exports = defineConfig({
transpileDependencies: true,
configureWebpack: {
plugins: [
new NodePolyfillPlugin()
]
}
})
Followed by npm install node-polyfill-webpack-plugin
I had this problem in ReactJS with create-react-app(facebook) but other packages (crypto-browserify)
Solution:
First install the necessary packages in this problem "path-browserify" but in my case "crypto-browserify"
Modify webpack.config.js in reactjs with create-react-app this file is inside:
node_modules/react-scripts/config/webpack.config.js
Search module.exports and inside this function there is a return:
module.exports = function (webpackEnv) {
...
return {
...
resolve: {
...
fallback: {
// Here paste
path: require.resolve("path-browserify"),
// But in mi case I paste
crypto: require.resolve("crypto-browserify"),
}
}
}
}
Note: This solution works, but when the webpack project starts it shows warnings
Pd: I am not native speaker English, but I hope understand me.
I have the same issue in React (feb-2022) using Real (realm-web) from Mongo.
I solve this in two steps:
npm i stream-browserify crypto-browserify
create fallback object into webpack.config.js at node_modules/react-scripts/config/webpack.config.js
module.exports = function (webpackEnv) {
...
return {
...
resolve: {
...
fallback: { // not present by default
"crypto": false,
"stream": false
}
}
}
I have the same issue with crypto. Some say that adding a proper path in TS config (and installing a polyfill) should resolve the issue.
tsconfig.json
{
"compilerOptions": {
"paths": {
"crypto": [
"./node_modules/crypto-browserify"
],
}
Details https://github.com/angular/angular-cli/issues/20819
I'm still fighting with crypto, but above link might help in your struggles.
I had the same issue, Strangely an import {script} from 'laravel-mix' in my app.js. I removed it and everything went back to normal.
I could not get the other answers to work with React on a new project with rss-parser. The error messages aren't clear. I fixed it by adding this line to the webpack configuration:
resolve: {
extensions: ['.tsx', '.ts', '.js'],
fallback: { "http": false, "browser": false, "https": false,
"stream": false, "url": false, "buffer": false, "timers": false
}
},
Watch out for this import in your app.js. Removing it fixed my issue.
use the version of webpack 4.46.0 to remove the error
I had to delete this line in a Vue component to solve the problem:
import { ContextReplacementPlugin } from "webpack";
If you use create-react-app(cra)
You have to overwrite the default webpack file
to do that you need to install a package like "react-app-rewired" that
will help overwriting and merge your configuration with the ones were
added by react scripts
First
npm install --D react-app-rewired
create a new file in the root directory called "config-overrides.js"
Install(using npm) and Add the required dependencies to the created config file
npm i -D path-browserify
install other packages as the error message shows
const webpack = require('webpack');
module.exports = function override(config) {
const fallback = config.resolve.fallback || {};
Object.assign(fallback, {
path: require.resolve('path-browserify'),
add others as shown in the error if there are more
});
config.resolve.fallback = fallback;
config.plugins = (config.plugins || []).concat([
new webpack.ProvidePlugin({
process: 'process/browser',
Buffer: ['buffer', 'Buffer'],
}),
]);
return config;
};
Update your package.json scripts
"scripts": {
"build": "react-app-rewired build",
....
"start": "react-app-rewired start",
"test": "react-app-rewired test"
},
Also please note that a new version of react-scripts(newer than 5.0.1) might solve the issue.
had this issue in a react-app.
installation of stream-browserify crypto-browserify didn't work for me
went through with hjpithadia's answer in Reactjs with a slight change in webpack.config.js
install the package
npm install node-polyfill-webpack-plugin
find the webpack configuration file here
node_modules/react-scripts/config/webpack.config.js
import the package
const NodePolyfillPlugin = require("node-polyfill-webpack-plugin");
search for module.exports
in plugin array add this new NodePolyfillPlugin(),
module.exports = function (webpackEnv) {
...
return {
...
plugins: [
new NodePolyfillPlugin(),
...
]
}
}
For React apps, if you don't need the module make sure webpack ignores it.
install craco https://craco.js.org/docs/getting-started/
create a craco.config.js in the root of your project, mine looks like this:
module.exports = {
webpack: {
configure: (webpackConfig) => {
webpackConfig.resolve.fallback = webpackConfig.resolve.fallback || {}
webpackConfig.resolve.fallback.zlib = false
webpackConfig.resolve.fallback.http = false
webpackConfig.resolve.fallback.https = false
webpackConfig.resolve.fallback.stream = false
webpackConfig.resolve.fallback.util = false
webpackConfig.resolve.fallback.crypto = false
return webpackConfig;
},
}
};
I resolved similar issue:
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "stream": false }
Installing the npm i stream-browserify --save-dev
Making the custom webpack.config.js file containing:
module.exports = {
resolve: {
fallback: { stream: require.resolve("stream-browserify") },
},
};
Installing the npm install #angular-builders/custom-webpack --save-dev
Assigning (angular.json: projects -> <proj_name> -> architect -> build -> options)
"customWebpackConfig": {
"path": "./webpack.config.js"
}
Changing the: build / serve / extract-i18n / test – as of your needs – to use custom Webpack config:
"builder": "#angular-builders/custom-webpack:browser",
I finally found the solution. With laravel, you have to change the webpack.mix.js file and not webpack.config.js.

Package.json exports with webpack 5 - dynamically imported module not found

I am having a bit of trouble reconciling the path of a dynamic import for i18n locales. Here's the relevant code -
function getLoader(
lang: SupportedLanguage,
ns: SupportedNamespace
): NamespaceLoader | undefined {
const matrixToCheck = UNSUPPORTED_MATRIX[ns];
const isSupported = matrixToCheck && matrixToCheck.indexOf(lang) === -1;
if (isSupported) {
const path = `./locales/${lang}/${ns}.json`;
const name = `${lang}_${ns}`;
const named = {
[name]: () => import(`${path}`),
};
return named[name];
}
}
...
// eventual output
const SUPPORTED_LANGUAGES = {en: {namespace1: () => import('./locales/en/namespace1.json')}
My goal is manage all of the relevant translations in a single npm package, handle all of the dynamic import set-up at build time, and then consumers can invoke the getter (getTranslation in this case) in their respectives apps for the language and namespace of their choice to get the payload at runtime.
Based on this GH thread, I wanted to reconcile the locale dist path via the package.json
...
"exports": {
".": "./dist/src/main.js",
"./": "./dist/"
},
...
e.g. when I publish the package, based on that exports config, the consumer would know know how to reconcile the path, either relative or package-name-prefix when the getter is invoked
const fn = () => import('./locales/fr/myNamespace.json') /// doesn't work
const anotherFn = () => import('#examplePackageName/locales/fr/myNamespace.json') /// doesn't work
Since everything is dynamic, I am using the CopyWebpackPlugin to include the locales in the dist folder.
This works as expected locally, but when I create the dist, I get the error Error: Module not found ./relative/path/to/the/json/I/want.json.
My question:
What am I missing? Is there a simple way to expose these translations so that other apps can include them in their bundles via an npm-installed package?
Here's my Webpack config, happy to provide other info as needed
const path = require("path");
const CopyPlugin = require("copy-webpack-plugin");
const { CleanWebpackPlugin } = require("clean-webpack-plugin");
const getPlugins = () => {
return [
new CleanWebpackPlugin(),
new CopyPlugin({
patterns: [{ from: "locales", to: "locales" }],
}),
];
};
module.exports = {
mode: "production",
entry: {
main: "./src/main.ts",
},
output: {
path: path.join(__dirname, "dist"),
filename: "src/[name].js",
chunkFilename: "chunk.[name].js",
libraryTarget: "commonjs2",
},
resolve: {
extensions: [".json", ".ts", ".js"],
alias: {
"#locales": path.resolve(__dirname, "locales/*"),
},
},
plugins: getPlugins(),
module: {
rules: [
{
test: /\.ts$/,
exclude: [/\.test\.ts$/],
include: path.join(__dirname, "src"),
loader: "ts-loader",
},
],
},
};
Exports directive prescribes to define all files allowed for import explicitly (documentation). It allows developer to hide internal package file structure. What's not exported by this directive is only available to import inside the package and not outside of it. It's made to simplify maintenance. It allows developers to rename files or change file structure without fear of breaking dependent packages and applications.
So if you want to make internal files visible for import, you should export them with exports directive explicitly, like this:
{
"exports": {
".": "./dist/esm/src/main.js",
"./dist/shared/locale/fr_fr.json": "./dist/shared/locale/fr_fr.json"
}
}
I'm not sure wether Webpack handling this case, because it's an experimental feature yet. But this is how Node.js works now.
Why it is so
Changing your app file structure is a major change in semver terms, so you need to bump a version everytime you rename or delete files. To avoid it you can specify which files are part of public interface of the package.

Node.js: How to import test files in custom test runner

I'm trying to create my own custom testing framework for learning purpose. Test files are written in following way
import { somemethod } from './some/module'
test(/I click on a button)/, () => {
browser.get("someSelector").should("have.text",somemethod());
});
I user require(file) to load test files. But it throw error SyntaxError: Unexpected token {
for import statement in test file. I'm using node js version 11.15.
If I switch to node v13.14 and define "type": "module" in my package.json then it doesn't let me use require(file) to load a test file or any module in my package.
How can I import tests files considering the user may be importing the modules using import or require?
This answer is very empirical...
Considering that it works using canonical commonjs approach you can try to debug it with newer version of NODE (currently I would use 14). For it, I would suggest you to use a node version manager like NVM so you can switch between node version easily and test that accordling seeing differences between various node installations.
Make a minimal project with npm init with a single dependency, save your index with the .mjs extension and try an import the above dependency. If you are be able to import that dependency with that minimal environment you can blame either your previous node or your configuration or both of them.
At the moment you should only create a small 2 files project to reproduce the problem. It seems your current node does not consider the "type": "module" configuration and runs everything in its classic way.
Regarding your comments....
As far as I know import can be used even in your code, not just at the beginning:
(async () => {
if (somethingIsTrue) {
// import module for side effects
await import('/modules/my-module.js');
}
})();
from https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import
Additionally you can try Webpack with a configuration like:
// webpack.config.js
const nodeExternals = require('webpack-node-externals');
module.exports = {
mode: 'production',
target: 'node',
externals: [nodeExternals()],
entry: {
'build/output': './src/index.js'
},
output: {
path: __dirname,
filename: '[name].bundle.js',
libraryTarget: 'commonjs2'
},
module: {
rules: [
{
test: /\.js$/,
use: {
loader: 'babel-loader',
options: {
presets: [
['env', {
'targets': {
'node': 'current'
}
}]
]
}
}
}]
}
};
With NodeExternals you don't put your node dependencies in the bundle but only your own code. You refer to node_modules for the rest. You might not want that.

Unit test for aws lambda using jest

const invokeApi = require("/opt/nodejs/kiwiCall");
const decrypt = require("/opt/nodejs/encryption");
const cors = require("/opt/nodejs/cors");
When I am testing my index.js file by manual mocking these dependencies in mocks directory as follows:
__mocks__
|_invokeApi
|_decrypt
|_cors
it says
FAIL ./index.test.js
● Test suite failed to run
Cannot find module '/opt/nodejs/kiwiCall' from 'index.js'
However, Jest was able to find:
'../../../../lambdas/Flights/Locations/index.js'
You might want to include a file extension in your import, or update your 'moduleFileExtensions', which is currently ['js', 'json', 'jsx', 'ts', 'tsx', 'node'].
See https://jestjs.io/docs/en/configuration#modulefileextensions-array-string
1 | "use strict";
2 |
> 3 | const invokeApi = require("/opt/nodejs/kiwiCall");
Wanted to know how can I mock the dependencies of AWS lambda in inedx.test.js file
In your package.json or jest.config you could add a moduleNameMapper for that directory.
"jest": {
"moduleNameMapper": {
"/opt/nodejs/(.*)": "<rootDir>/../nodejs/$1"
},
},
So I managed to figure out something based on my repository.
I'm using the moduleNameMapper to map the absolute path to another location in my repository to where I have the layer stored.
Eg.
moduleNameMapper: {'^/opt/config/config': '<rootDir>/src/layers/layers-core/config/config'}
In your case you could use a regex expression to match /opt/nodejs/ and map it elsewhere. Hope that helped.
EDIT:
I completely changed my approach and used babel-plugin-module-resolver with babel-rewire. I did this because the above method was incompatible with rewire. It's quite easy setup and you just need to setup a babel alias within .babelrc.
eg.
{
"plugins": [
["rewire"],
["babel-plugin-module-resolver", {
"alias": {
"/opt/config/config": "./src/layers/layers-core/config/config",
"/opt/utils/util-logger": "./src/layers/layers-core/utils/util-logger",
"/opt/slack": "./src/layers/layers-slack/slack"
}
}]
]
}
Combine this with IDE jsconfig.json path alias and you get full IDE support.
{
"compilerOptions": {
"module": "commonjs",
"target": "es2018",
"baseUrl": "./",
"paths": {
"/opt/config/config": ["src/layers/layers-core/config/config"],
"/opt/utils/util-logger": ["src/layers/layers-core/utils/util-logger"],
"/opt/slack/*": ["src/layers/layers-slack/slack/*"],
}
},
"exclude": ["node_modules", "dist"]
}
You can then reference your layers with jest.doMock('/opt/config/config', mockConfig);
EDIT 2:
Found a way to get Jest to mock it. Just slip {virtual: true} into the mock!
jest.doMock('/opt/config/config', mockConfig, {virtual: true});
I have pretty much the same issue. I have defined a layer which contains common code that's shared between other functions in my project. My project structure looks something like this:
project/
functions/
function1/
app.js
function2/
app.js
shared/
shared.js
I import my shared library like this:
const { doSomething } = require('/opt/shared');
exports.handler = async (event) => {
const result = await doSomething();
// etc...
return {statusCode: 200};
}
This works when I deploy to AWS Lambda because the /opt/shared exists and it can be referenced correctly. It also works if I run this on my machine using sam local invoke Function1 because it's running in a container, which makes /opt/shared available to the code.
However, I'm struggling to work out how I can mock this dependency in a unit test. If I simply do this: jest.mock('/opt/shared'), I'm getting: Cannot find module '/opt/shared' from app.test.js
You can use the modulePaths option, from this post.
Documentation
jest.config.js
"jest": {
"modulePaths": [
"<rootDir>/src/layers/base/nodejs/node_modules/"
]
}
You can dynamically create this array by scanning a directory
const testFolder = './functions/';
const fs = require('fs');
const modulePaths = fs.readdirSync(testFolder)
.reduce((modulePaths, dirName) => {
modulePaths.push(`functions/${dirName}/dependencies/nodejs/node_modules/`);
return modulePaths;
}, []);

Webpack import from files that use module.exports

I have React app and a file where I want to store things related to api.
const proxy = require('http-proxy-middleware');
const path = require('path');
//.....
const targetApi = (objectWithUrlEntries) => {
Object.keys(objectWithUrlEntries).forEach((key) => {
objectWithUrlEntries[key] = path.join('/api/', objectWithUrlEntries[key]);
});
};
module.exports.proxyExpressCalls = proxyExpressCalls;
module.exports.devServerProxyConfig = devServerProxyConfig;
module.exports.targetApi = targetApi;
Some of those things will be used by webpack itself, and some will be used inside the app (to correctly target api calls).
However when I try to import things in my app:
// #flow
import { buildUrl } from 'data/utils';
import type { Axios } from './flow.types';
import { targetApi } from './api';
console.log(targetApi());
I get errors. In terminal:
WARNING in ./src/data/redux/api/user.js 6:12-21 "export 'targetApi'
was not found in './api'
in browser:
api.js?d669:39 Uncaught TypeError: Cannot set property 'proxyExpressCalls' of undefined
at Object.eval (api.js?d669:39)
at eval (api.js:60)
at Object../src/data/redux/api/api.js (client.bundle.js:11620)
at __webpack_require__ (client.bundle.js:708)
at fn (client.bundle.js:113)
at eval (user.js:15)
at Object../src/data/redux/api/user.js (client.bundle.js:11668)
at __webpack_require__ (client.bundle.js:708)
at fn (client.bundle.js:113)
at eval (user.js:18)
So the problem is that when app is being bundled commonjs exports fail, but if I would use es6 export syntax then Node would fail.
I had a similar problem: I had a javascript class with some validation rules that I wanted to use in Node JS and also in the client code. What worked for me was converting everything to Common JS, the shared code, the node code, and the client code. But I still had some problems. Then I added "module": "commonjs" to my .babelrc of the folder that imports the shared code and it finally worked. This is my .babelrc file:
{
"presets": [
"react",
[
"env",
{
"debug": true,
"modules": "commonjs",
"targets": {
"browsers": [
"last 2 versions",
"safari >= 7"
],
}
}
],
],
"plugins": [
"transform-object-rest-spread",
"transform-es2015-arrow-functions",
"transform-class-properties"
]
}
Another possible solutions is (not tested!) to create a library out of your shared code, using webpack. Check the output.library and output.libraryTarget options to see which options you have to expose the library in different module systems. Then import your shared library in your node and client code.
The browser error holds the key: it looks like module.exports is null. And sure enough, you're setting values on it but it was not initialized. If instead you do this:
module.exports = {
proxyExpressCalls: proxyExpressCalls,
devServerProxyConfig: devServerProxyConfig,
targetApi: targetApi
};
(or simply set module.exports = {} first) this should solve the problem. The console warning is likely a side effect of code that keeps going even after the failure to set values on a null variable.

Resources