I have a hybrid cjs/esm Node package written in Typescript.
It consists of, let's say, two files - core.ts and extra.ts and I want to keep them separate. extra.ts imports core.ts (literally import { ... } from './core';) and they are separate entry points of the package. Clients import either only core.ts or both.
Rollup makes it easy to have build steps for multiple entry points, for multiple output formats, and all seemed fine.
But now I ran into an issue:
Let's say I have example.mjs or example.cjs where I import or require my-package and my-package/extra entry points.
That doesn't work. Error messages are slightly different but meaning is the same - ./core module can not be found when reading extra.mjs/cjs.
extra.cjs built by Rollup contains the line var core = require('./core');
extra.mjs built by Rollup contains the line import { ... } from './core';
By default, Node 12 does not guess file extensions (I'm not questioning this).
I have to call it as node --experimental-modules --es-module-specifier-resolution=node ./example.mjs to make it work. This is unsatisfactory solution. I need example.mjs to be runnable without additional flags.
It appears to me that file extensions can and should be added to import/require statements in compiled cjs/mjs files to make it work according to the spec.
Although, since I have different build steps in Rollup config for both files and external: ['./core'] in the options for extra.ts, for Rollup they are totally unrelated. And Rollup will just bundle them into a single file otherwise, which is not what I need either.
So the question:
Is there a plugin or a configuration option to make Rollup produce files with correct local imports (file extensions added to local imports according to the format)?
What would be the best way to add an extra step to the building process to patch imports if there is no existing solution?
Maybe there a different bundler that might work for the same task?
Got a satisfactory solution.
output.preserveModules option keeps all files separate while building them in one pass (can be limiting in some aspects though).
output.entryFileNames option allows to specify file extensions. (It won't work with modules considered external, can't use it without preserveModules.)
I can build only extra.ts as it imports every other file. But I have to be mindful of tree shaking when building like this - need to retain all exports of core.ts.
export default [
{
external: [],
input: 'src/extra.ts',
treeshake: false,
plugins: [
typescript(),
cleanup({ extensions: ['ts'] })
],
output: [
{
dir: 'lib',
format: 'es',
preserveModules: true,
entryFileNames: '[name].mjs',
},
{
dir: 'lib',
format: 'cjs',
preserveModules: true,
entryFileNames: '[name].cjs',
},
],
},
];
One step back is that I get another output file for another ts file imported by core.ts and extra.ts. I think I can live with that.
Ideal solution would require to monkey-patch output files with sed-like tool after the build in my initial configuration.
Related
Architecture
I would like to share code between client and server side. I have defined aliases in the webpack config:
resolve: {
// Absolute paths: https://github.com/webpack/webpack/issues/109
alias: {
server : absPath('/src/server/'),
app : absPath('/src/app/'),
client : absPath('/src/client/'),
}
},
Problem
Now on the server side I need to include webpack in order to recognize the correct paths when I require a file. For example
require('app/somefile.js')
will fail in pure node.js because can't find the app folder.
What I need (read the What I need updated section)
I need to be able to use the webpack aliases. I was thinking about making a bundle of all the server part without any file from node_modules. In this way when the server starts it will use node_modules from the node_modules folder instead of a minified js file (Why? 1st: it doesn't work. 2nd: is bad, because node_modules are compiled based on platform. So I don't want my win files to go on a unix server).
Output:
Compiled server.js file without any node_modules included.
Let the server.js to use node_modules;
What I need updated
As I've noticed in https://github.com/webpack/webpack/issues/135 making a bundled server.js will mess up with all the io operation file paths.
A better idea would be to leave node.js server files as they are, but replace the require method provided with a custom webpack require which takes in account configurations such as aliases (others?)... Can be done how require.js has done to run on node.js server.
What I've tried
By adding this plugin in webpack
new webpack.optimize.CommonsChunkPlugin(/* chunkName= */"ignore", /* filename= */"server.bundle.js")
Entries:
entry: {
client: "./src/client/index.js",
server: "./src/server/index.js",
ignore: ['the_only_node_module'] // But I need to do that for every node_module
},
It will create a file server.js which only contains my server code. Then creates a server.bundle.js which is not used. But the problem is that webpack includes the webpackJsonp function in the server.bundle.js file. Therefore both the client and server will not work.
It should be a way to just disable node_modules on one entry.
What I've tried # 2
I've managed to exclude the path, but requires doesn't work because are already minified. So the source looks like require(3) instead of require('my-module'). Each require string has been converted to an integer so it doesn't work.
In order to work I also need to patch the require function that webpack exports to add the node.js native require function (this is easy manually, but should be done automatically).
What I've tried # 3
In the webpack configuration:
{target: "node"}
This only adds an exports variable (not sure about what else it does because I've diffed the output).
What I've tried # 4 (almost there)
Using
require.ensure('my_module')
and then replacing all occurrences of r(2).ensure with require. I don't know if the r(2) part is always the same and because of this might not be automated.
Solved
Thanks to ColCh for enlighten me on how to do here.
require = require('enhanced-require')(module, require('../../webpack.config'));
By changing the require method in node.js it will make node.js to pass all requires trough the webpack require function which allow us to use aliases and other gifts! Thanks ColCh!
Related
https://www.bountysource.com/issues/1660629-what-s-the-right-way-to-use-webpack-specific-functionality-in-node-js
https://github.com/webpack/webpack/issues/135
http://webpack.github.io/docs/configuration.html#target
https://github.com/webpack/webpack/issues/458
How to simultaneously create both 'web' and 'node' versions of a bundle with Webpack?
http://nerds.airbnb.com/isomorphic-javascript-future-web-apps/
Thanks
Thanks to ColCh for enlighten me on how to do here.
require = require('enhanced-require')(module, require('../../webpack.config'));
By changing the require method in node.js it will make node.js to pass all requires trough the webpack require function which allow us to use aliases and other gifts! Thanks ColCh!
My solution was:
{
// make sure that webpack will externalize
// modules using Node's module API (CommonJS 2)
output: { ...output, libraryTarget: 'commonjs2' },
// externalize all require() calls to non-relative modules.
// Unless you do something funky, every time you import a module
// from node_modules, it should match the regex below
externals: /^[a-z0-9-]/,
// Optional: use this if you want to be able to require() the
// server bundles from Node.js later
target: 'node'
}
I'm working on an application built with Brunch. I would like to load some* of the vendor-supplied javascript as modules, so that I can require them in my code, rather than relying on global variables. Is there some way to do this, without copying all the vendor code into my app directory?
I tried creating a vendorlib directory, but brunch doesn't seem to look anywhere bu app and vendor. I also tried making a vendor/modules directory, but brunch seems to not wrap anything found under vendor (even when I convinced it to combine those files with the files other modules found under app.)
*The "some" that I'm working on right now are Chaplin, Backbone and Underscore. If I get those to work, I'll move more over later.
You can override config.modules.wrapper and make it wrap, for example, all files in vendor/modules directory. Or you can make add more directories that are handled by brunch to config.paths.watched.
For those following along at home, this is what my config.coffee eventually looked like:
paths:
watched: ['app','vendor','test','vendorlib']
files:
javascripts:
joinTo:
'javascripts/app.js': /^app/
'javascripts/vendor.js': /^vendor/
'test/javascripts/test.js': /^test[\\/](?!vendor)/
'test/javascripts/test-vendor.js': /^test[\\/](?=vendor)/
order:
# Files in `vendor` directories are compiled before other files
# even if they aren't specified in order.before.
before: [
'vendor/scripts/console-polyfill.js',
]
after: [
'test/vendor/scripts/test-helper.js'
]
stylesheets:
joinTo:
'stylesheets/app.css': /^(app|vendor)/
'test/stylesheets/test.css': /^test/
order:
after: ['vendor/styles/helpers.css']
templates:
joinTo: 'javascripts/app.js'
modules:
nameCleaner: (path) ->
path.replace(/^(app|vendorlib)\//, '')
This lets me populate a vendorlib directory with modules from vendors that support loading as modules. I currently have Chaplin, jQuery, and Backbone in there. I had to rename them not to include the version numbers.
I am trying to integrate the r.js optimizer on the server side (Apache Sling) and face one problem: when resolving modules it always looks them up under the output directory (dir), not from within the source directory (baseUrl or appDir), doesn't find them and thus fails.
/project/build.js
({
name: "modules/main",
dir: "/target",
baseUrl: "/sources"
})
If you wonder, the root path / is inside the server's JCR repository, not a file system. Also I simplified the example a bit (hopefully without concealing the issue).
It will resolve and read the main file properly:
/sources/modules/main.js
require(["modules/foo"]);
However, when it now tries to resolve modules/foo, it tries to read it from /target/modules/foo.js instead of /sources/modules/foo.js as I would expect, which does not exist and the whole r.js execution fails and stops.
I tried using appDir and all kinds of combinations, but the issue is always the same. I am fairly sure it is not related to my integration code... AFAIU from documentation and googling around, it should either copy them to the target before building the optimized file or simply pick them up from the source directory automatically.
Am I supposed to copy all the raw source files to /target myself before running r.js?
Maybe the problem is that baseUrl=/overlay is different from build.js residing inside /project?
Maybe r.js also looks at the current working directory of the r.js process (which is so far undefined in my case)?
Can the output directory (dir) live outside appDir or baseUrl?
My require.js configuration looks like so:
({
appDir: "../app",
baseUrl: "js/lib", // means the base URL is ../app/js/lib
dir: "../app-built", //target
// offtopic, but a very handy option
mainConfigFile: "../app/config.js",
// I'm not 100% sure if it's equivalent to your version
// where you're not using "modules" and just "name"
modules: [{
name: "../some/main" // this is ../app/js/some/main.js
}]
})
Reading through https://github.com/jrburke/r.js/blob/master/build/example.build.js#L15 - it seems you do want an appDir specified if you want the files to be copied to the target dir before optimization.
To answer your other questions
you don't need to manually copy files over
baseUrl should point to the same place as baseUrl used in your app's config - however you have to adjust it depending on what appDir you choose to use (e.g. appDir="../app" and baseUrl="js/lib", or appDir="../app/js" then baseUrl="lib", etc.)
appDir and dir should be relative to the build config file - I don't know what happens when you use absolute paths
yes - output dir does (has to?) live outside appDir. BaseURL is within the appDir/dir (all these names are really confusing..)
I would say
use the "appDir" setting
try using "modules" like I did instead of just "name"
make "appDir" and "dir" relative paths to the build file if you can - these absolute paths might be what's breaking? because other than that the config looks very similar to the one I use
I know there's a different way of configuring it where your output is 1 file, which case the files are read from the source dir - but I haven't used that much myself.
Hope this helps.
Answering myself: I got it to work with the single output file approach using out instead of appDir or dir:
({
name: "modules/main",
baseUrl: "/sources"
out: "/target/out.js",
})
In this case it reads all the modules from the sources and creates a /target/out-temp.js which it then moves to /target/out.js when done.
This seems to suit my needs so far.
If I use Modules options, I see the whole files are copied and I am not sure how to use a single optimized file.
I can build a file with references to JS files and use that file to generate a single file.
But using Modules option, I see only the source files are copied directly and I see no optimization....any idea on how to use modules option and optimize it?
Here is the snippet...
({
baseUrl: ".",
dir: "app/main-built-125",
modules: [{
name: "scripts/Address/AddressList"
}, {
name: "scripts/Office/OfficeDetails"
}],
paths: {
jquery: 'empty'
},
fileExclusionRegExp: /^\./,
})
You'll need to use the include option to specify which dependencies to include with the module.
Note that by default dependencies of the defined modules (the one set as name) should be fetched and included in the builded file (Of course, as long as they're defined in the define() deps array).
I have code like the following
define("ModuleA", ["InitialDependency"], function (initDep){
return {};
});
define("ModuleB", ["ModuleA", "OtherDependency"], function (moduleA, otherDep){
return {};
});
Each of these modules is defined in separate files "ModuleA.js", "Moduleb.js", "InitialDependency.js" and "OtherDependency.js".
These modules are loaded sequentially in my application. ModuleB is always loaded after ModuleA. this means that in the optimization stage I do not want ModuleA's script combined in the built script for ModuleB. I want the following
ModuleA.built.js includes
InitialDependency
ModuleA
ModuleB.built.js includes
OtherDependency
ModuleB
I don't want them all in the same file however as ModuleB may never be loaded.
I can do a build script for both modules but this will be time consuming as I have quite a few modules in my project and would like a build script that will build the lot of them at once.
What do I need to know to create a build script for building both of these modules (and more that follow the same dependency pattern)?
To achieve this, you'd have to play with the modules configuration option.
It could look like this:
{
modules: [
{
name: "ModuleA",
include: [],
exclude: []
},
{
name: "ModuleB",
exclude: [
"moduleA"
]
}
]
}
There's a similar example setup by James here: https://github.com/requirejs/example-multipage
Of course, by building these modules separately, you may end up needing to update paths. If so, the best way then would be to create a file containing a require.config call with special setting for your builded app and including this configuration instead of your usual one. But if you set dependencies in a good separated way, then you'll probably be fine. By "good separated" way, I mean that if moduleA is the base script, then it shouldn't have dependencies packed with moduleB - but I guess this is common sense!
Note about shimmed modules: As shimmed config only work whe files are loaded and by r.js to order plugins, be sure you don't include a shim module without it's dependency if you're not 100% sure these will be loaded before. More info here: https://github.com/requirejs/example-multipage-shim
Hope this help!