#loadable/server pass the whole stats JSON to eval('require')(modulePath) - node.js

I'm trying to setup SSR for react app with #loadable/components. I setup all based on docs with babel and webpack plugins. When I try to run node server.js it runs ok but when I open a browser and throws the following error (into node console):
TypeError [ERR_INVALID_ARG_TYPE]: The "id" argument must be of type string. Received an instance of Object
at validateString (internal/validators.js:118:11)
at Module.require (internal/modules/cjs/loader.js:1033:3)
at require (internal/modules/cjs/helpers.js:72:18)
at smartRequire (/Users/max/Documents/repos/app/node_modules/#loadable/server/lib/util.js:44:25)
at new ChunkExtractor (/Users/max/Documents/repos/app/node_modules/#loadable/server/lib/ChunkExtractor.js:181:50)
at renderer (webpack://app/./node_modules/#MYSCOPE/elm/dist/elm.esm.js?:3619:19)
at eval (webpack://app/./src/server.tsx?:64:90)
at processTicksAndRejections (internal/process/task_queues.js:97:5) {
code: 'ERR_INVALID_ARG_TYPE'
}
As you can see there is #MYSCOPE in the traceback which holds some of my internal packages (if it matters).
#loadable/server/lib/util.js is the following function:
And when I try to console.log(modulePath) on line 42 I see a whole stats JSON output which seems wrong and I should get a single module path (as I understand).
Any help?
I can share some specific parts of my configuration files if needed. Because I see my own package in console output seems like something is wrong with it's build (it works perfectly on the client-side with cjs build), but having full stats object as module path is very confusing.
UPD: Demo https://www.dropbox.com/s/9r947cgg4qvqbu4/loadable-test.zip?dl=0
Run
yarn
yarn dev:server
# go to localhost:3000 and see the error in console
to rebuild:
yarn
yarn dev:build-client
yarn dev:build-server
yarn dev:server # go to localhost:3000

The statsFile option passed to ChunkExtractor expects a path to the loadable-stats.json file, not the actual JSON content of it. By doing require('../loadable-stats.json'), webpack actually resolve the JSON during build time and assign it to the loadableJson variable.
You can change your loadableJson as follow:
import path from 'path';
const loadableJson = path.resolve(__dirname, '../bundle_client/loadable-stats.json');
This will solve the problem you had on your question. But, if you only do this, you will notice that you have another problem. Loadable by default assumes that your entry chunk name is main. This is not the case in your demo, as you have set the entry chunk name to be app instead.
entry: {
app: ['#babel/polyfill', './src/index.tsx']
},
To solve this, simply tell loadable about your entrypoints names by passing an array to the ChunkExtractor contructor as such:
const extractor = new ChunkExtractor({
statsFile: loadableJson,
entrypoints: ["app"], // name of your entry chunk
});
That's it, everything should now be working properly!
If it helps, I set up the demo on GitHub so you can easily see the changes I made here.

Related

Node JS (cpanel) Error: I'm getting an error [ERR_REQUIRE_ESM]: Must use import to load ES Module

So, I've spent quite a few hours today trying to put my nodeJS app that's fully using ESM (modules), and I've deployed it via cPanel on a server that's using Node v. 14.20.1. I'm constantly getting an error:
App 1153856 output: internal/modules/cjs/loader.js:948
App 1153856 output: throw new ERR_REQUIRE_ESM(filename);
App 1153856 output: ^
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /<serverlocation>/app.js
App 1153856 output: at new NodeError (internal/errors.js:322:7)
App 1153856 output: at Module.load (internal/modules/cjs/loader.js:948:11)
App 1153856 output: at Function.Module._load (internal/modules/cjs/loader.js:790:12)
App 1153856 output: at Module.require (internal/modules/cjs/loader.js:974:19)
App 1153856 output: at Module.require (/opt/cpanel/ea-ruby27/root/usr/share/passenger/helper-scripts/node-loader.js:80:25)
All files are written as modules, I don't have one "require()" anywhere.
Since the Node started fully supporting JS modules from v14 on, I'm taking a wild guess that the hosting server I'm using (and their Passenger for NodeJS) is using a loader that's using "require()" when calling my app.js file.
I've tried multiple solutions, I've even switched my app.js file to a CommonJS type, but then it required me to switch all other files to CJS as well, which would be too much hassle.
Has anyone managed to find a proper solution to this issue?
For anyone trying to solve this issue, this is how I solved it:
1- Create a loader script: Not necessarily in the same folder as the main app.js file of your app, but that's where I created it. Call it something like loader.cjs. The extension being .cjs is important if you have "type": "module" in your package.json.
As you might've guessed, this will be the new main of your app. Since the passenger's loaders have an issue with ES modules, just let it load a commonjs file instead.
2- Dynamic import of app.js: Did you know that you can still load ES modules in commonjs files? You just need a little extra bit to do so.
Apparently, ES modules are loaded asynchronously, and this doesn't work well with synchronous commonjs files. That's why you got the issue in the first place, right?
Therefore, the solution is: dynamic imports. Just like async functions, treat the imports of ES modules as promises. I don't really like using .then(), so I opted with await:
async function loadApp() {
await import("/path/to/app.js");
}
loadApp();
3- Rename your app.js's extension: I don't exactly know why this is necessary, but I got error along the lines of "couldn't find /path/to/app.mjs" and so I changed it to so. Then it worked. You can keep the name of the file as "app.js" in the path in the previous point, and the import will sstill correctly look for "app.mjs";
There might be more efficient ways to do this, but that's what my 2 bits of brain could come up with. Hopefully it helps others as well.

Error: Identifier 'Reader' has already been declared while bundling fstream with rollup.js

I'm new with configuring stuff with Rollup, I'm trying to configure a node CLI app to be bundled in a single cli.bundle.js file that would be executable with a simple node cli.bundle.js without needing to npm install anything before.
On a side note, we already have succeed in doing so with zeit/pkg, but we would rather note have all the nodejs executable bundled inside, so we are trying work with rollup instead of pkg.
The problem we encounter is that when going through the different node_modules of the application, rollup.js crash with error:
[!] Error: Identifier 'Reader' has already been declared
../../common/js-common/node_modules/fstream/lib/file-reader.js (7:4)
5: var fs = require("graceful-fs")
6: , fstream = require("../fstream.js")
7: , Reader = fstream.Reader
^
8: , inherits = require("inherits")
9: , mkdir = require("mkdirp")
Error: Identifier 'Reader' has already been declared
at error (/home/.../src/external-data/external-data-etl/node_modules/rollup/dist/shared/node-entry.js:5400:30)
at Module.error (/home/.../src/external-data/external-data-etl/node_modules/rollup/dist/shared/node-entry.js:9820:16)
at tryParse (/home/.../src/external-data/external-data-etl/node_modules/rollup/dist/shared/node-entry.js:9713:23)
at Module.setSource (/home/.../src/external-data/external-data-etl/node_modules/rollup/dist/shared/node-entry.js:10076:33)
at Promise.resolve.catch.then.then.then (/home/.../src/external-data/external-data-etl/node_modules/rollup/dist/shared/node-entry.js:12362:20)
While looking for this error with rollup, it seems people were having it more at execution time than at bundle time, so I have no clue of what I can do. This duplicated identifier is in a 3rd party code I don't control :(
Here is my rollup.config.js
Edit: I tried with the new #rollup/plugins to see if there were a fix in them, but I have still the same issue.
import commonjs from '#rollup/plugin-commonjs';
import resolve from '#rollup/plugin-node-resolve';
import json from '#rollup/plugin-json';
export default {
input: 'dist/index.js',
output: {
format: 'cjs',
file: './cli.bundle.js'
},
plugins: [
commonjs(),
resolve(),
json() // asked and added when parsing 'got' package imported by 'download' package
]
};
And our building process is :
transpile from typescript (src) to js (dist) with tsc
bundle (dist) app into single runnable file
We would rather not include babel or typescript plugin to transpile, to stay independant, and certainly the (dist) app is enough (as it was enough for zeit/pkg).
Is there something we are doing wrong ?
The Problem is your dependency it is not coded in the right way you will need to correct the code error they did.

How do I obfuscate a JS file with nightmare.js library

When I package the electron app on macOS I can never obfuscate the file with Nightmare because of its limitations. Would I need to re-write the whole library, or is there a way I can get around this?
There is a solution. Absolutely unstable and untested, try at your own risk. And ofc, if you can find and fix the bugs that comes as side effect, feel free to share them :)
We are going to use a module called pkg to bundle up the script with node. Also for simplicity, we will use npx.
There were some side-effects but that was first time something worked with electron and nightmare.
Consider the following script,
const Nightmare = require("nightmare");
const nightmare = Nightmare({
show: true
});
nightmare
.goto("https://example.com")
.title()
.end()
.then(console.log)
.catch(error => {
console.error(error);
});
This is a simple script that says, go to example.com and give me the title.
Cool! Let's try using it thru npx and pkg. The code for that is,
npx pkg app.js --target 'host'
However, we got some nasty errors,
> Warning Cannot include file %1 into executable.
The file must be distributed with executable as %2.
node_modules/nightmare/lib/runner.js
path-to-executable/nightmare/runner.js
> Warning Cannot include file %1 into executable.
The file must be distributed with executable as %2.
node_modules/nightmare/lib/frame-manager.js
...
etc etc. and the file wouldn't run.
Error: spawn /home/someone/Desktop/a/electron/dist/electron ENOENT
It cannot find the required files as they were not bundled. We will use process.cwd() to get those data which resides in relevant folder.
const nodeDir = process.cwd() + "/node_modules/"; // <- Get node modules folder
const nightmareDir = nodeDir + "nightmare"; // <-- Get nightmarejs path
const electronDir = nodeDir + "electron"; // <-- Get electron path
const Nightmare = require(nightmareDir);
const electronPath = require(electronDir);
const nightmare = Nightmare({
show: true,
electronPath // <-- use the specific electron path
});
nightmare
.goto("https://example.com")
.title()
.end()
.then(console.log)
.catch(error => {
console.error(error);
});
When I ran it, it showed me some more warning, but that's because I did not optimize the process.cwd() yet. Then I ran it and voila!
➜ a npx pkg app.js --target 'host'
> pkg#4.3.1
> Warning Cannot resolve 'nightmareDir'
/home/someone/Desktop/a/app.js
Dynamic require may fail at run time, because the requested file
is unknown at compilation time and not included into executable.
Use a string literal as an argument for 'require', or leave it
as is and specify the resolved file name in 'scripts' option.
> Warning Cannot resolve 'electronDir'
/home/someone/Desktop/a/app.js
Dynamic require may fail at run time, because the requested file
is unknown at compilation time and not included into executable.
Use a string literal as an argument for 'require', or leave it
as is and specify the resolved file name in 'scripts' option.
➜ a ./app
Example Domain // <-- Our sweet result :D
➜ a
This can be improved and tweaked, but I'll leave that to you.

Getting a directory outside symbolic links when running script nodejs

Let's assume the following example
ROOT
config
my.json
releases
today
index.js
current (symlink to today for example)
i would like to include the json file in my index.js file using a simple requirestatement, require('my.json')
The nodejs script would be launcher using node index.js or node fullpath/index.js
The problem encountered is that the require path will always resolve to releases/today/index.js instead of current/index.js. Because of this the inclusion of my json file is not correct.
The following techniques have been tried, followed by the output:
__dirname: /ROOT/releases/today
process.cwd(): /ROOT/releases/today
process.env.PWD: depending on the startup location of the script: / or /ROOT/current
require('path').resolve(__dirname)): /ROOT/releases/today
process.env.PWD doesn't always give stable results and i was wondering if there is something that has the same result.

How to not bundle node_modules, but use them normally in node.js?

Architecture
I would like to share code between client and server side. I have defined aliases in the webpack config:
resolve: {
// Absolute paths: https://github.com/webpack/webpack/issues/109
alias: {
server : absPath('/src/server/'),
app : absPath('/src/app/'),
client : absPath('/src/client/'),
}
},
Problem
Now on the server side I need to include webpack in order to recognize the correct paths when I require a file. For example
require('app/somefile.js')
will fail in pure node.js because can't find the app folder.
What I need (read the What I need updated section)
I need to be able to use the webpack aliases. I was thinking about making a bundle of all the server part without any file from node_modules. In this way when the server starts it will use node_modules from the node_modules folder instead of a minified js file (Why? 1st: it doesn't work. 2nd: is bad, because node_modules are compiled based on platform. So I don't want my win files to go on a unix server).
Output:
Compiled server.js file without any node_modules included.
Let the server.js to use node_modules;
What I need updated
As I've noticed in https://github.com/webpack/webpack/issues/135 making a bundled server.js will mess up with all the io operation file paths.
A better idea would be to leave node.js server files as they are, but replace the require method provided with a custom webpack require which takes in account configurations such as aliases (others?)... Can be done how require.js has done to run on node.js server.
What I've tried
By adding this plugin in webpack
new webpack.optimize.CommonsChunkPlugin(/* chunkName= */"ignore", /* filename= */"server.bundle.js")
Entries:
entry: {
client: "./src/client/index.js",
server: "./src/server/index.js",
ignore: ['the_only_node_module'] // But I need to do that for every node_module
},
It will create a file server.js which only contains my server code. Then creates a server.bundle.js which is not used. But the problem is that webpack includes the webpackJsonp function in the server.bundle.js file. Therefore both the client and server will not work.
It should be a way to just disable node_modules on one entry.
What I've tried # 2
I've managed to exclude the path, but requires doesn't work because are already minified. So the source looks like require(3) instead of require('my-module'). Each require string has been converted to an integer so it doesn't work.
In order to work I also need to patch the require function that webpack exports to add the node.js native require function (this is easy manually, but should be done automatically).
What I've tried # 3
In the webpack configuration:
{target: "node"}
This only adds an exports variable (not sure about what else it does because I've diffed the output).
What I've tried # 4 (almost there)
Using
require.ensure('my_module')
and then replacing all occurrences of r(2).ensure with require. I don't know if the r(2) part is always the same and because of this might not be automated.
Solved
Thanks to ColCh for enlighten me on how to do here.
require = require('enhanced-require')(module, require('../../webpack.config'));
By changing the require method in node.js it will make node.js to pass all requires trough the webpack require function which allow us to use aliases and other gifts! Thanks ColCh!
Related
https://www.bountysource.com/issues/1660629-what-s-the-right-way-to-use-webpack-specific-functionality-in-node-js
https://github.com/webpack/webpack/issues/135
http://webpack.github.io/docs/configuration.html#target
https://github.com/webpack/webpack/issues/458
How to simultaneously create both 'web' and 'node' versions of a bundle with Webpack?
http://nerds.airbnb.com/isomorphic-javascript-future-web-apps/
Thanks
Thanks to ColCh for enlighten me on how to do here.
require = require('enhanced-require')(module, require('../../webpack.config'));
By changing the require method in node.js it will make node.js to pass all requires trough the webpack require function which allow us to use aliases and other gifts! Thanks ColCh!
My solution was:
{
// make sure that webpack will externalize
// modules using Node's module API (CommonJS 2)
output: { ...output, libraryTarget: 'commonjs2' },
// externalize all require() calls to non-relative modules.
// Unless you do something funky, every time you import a module
// from node_modules, it should match the regex below
externals: /^[a-z0-9-]/,
// Optional: use this if you want to be able to require() the
// server bundles from Node.js later
target: 'node'
}

Resources