I have been absent from nodejs development for a few years, and to my surprise when I did
$ npm install lodash
the size of my node_modules grew by 4.8MiB
how come a library that judging by its source line count should be around 260KiB takes so much space?
I am deploying my code as an AWS Lambda, and the ability to edit and debug in the console is lost due the increased size of my distribution file (partly because of this).
What's the 2019 way to deal with this?
You node_modules size is irrelevant.
If you build you application the size is much much smaller:
minified 69.2kb
minified + gzipped 24.3kB
see here
If you're using Webpack, ESBuild, or a similar bundler with tree-shaking capabilities, you can import lodash functions discretely.
import foo from 'lodash/foo';
Support for this syntax was added in Lodash v4.
Note the syntax should be 'lodash/foo'. Contrarily, import { foo } from 'lodash' will not be tree-shaken.
Like Norbert said, your node_modules size is irrelevant. You'll want to look at the size of your bundle after building your application. In the case of Webpack, you can use Webpack Bundle Analyzer to gain additional insights into your bundle's distribution. For Vite and other toolchains that use Rollup, consider rollup-plugin-visualizer.
Here's a few more tips:
As of lodash v4, tree-shaking works without additional configuration in Webpack >= v4 and ESBuild. If you use lodash-es and you have other dependencies that require lodash, both will end up in your bundle.
If you're using Babel, and you don't want to refactor your entire codebase to use the aforementioned lodash/foo syntax, consider using babel-plugin-lodash. This plugin rewrites lodash import statements to use the tree-shaking syntax for you.
If you're using Babel's preset-env plugin, set the modules option to false. Babel rewrites modules to use CommonJS by default, which won’t tree-shake.
If you're only using a single Lodash function, you can install it as a standalone module in lieu of installing the entire lodash package. e.g. npm install --save lodash.functionName
lodash-es introduces a few exceptions to this strategy. Refer to this answer for more insight into this.
Related
I'm using WebStorm and a newbie to it. when I use import/export statements, it gives me an error of
Unexpected token import
but if I try with require/module.exports it works fine.
N.B- I've configured language version as ES6 from Languages and Frameworks.
This is not WebStorm but Node.js that fails. While import is a part of ES6, native support for ES6 modules in Node.js is very limited and requires special setup - see https://nodejs.org/api/esm.html#esm_enabling. So, you have to compile your code with Babel first. Usually transpiling is a part of build process (using Gulp, Grunt, WebPack, etc.). Or, you can transpile your code on-the-fly by passing -r babel-register to Node.js. Of course, you need creating appropriate .babelrc and install the required modules (npm install --save-dev babel-cli babel-preset-env)
I am trying to use CommonJS module format, with create-react-app (CRA). The documentation for CRA says:
..Create React App can consume both CommonJS and ES modules. For Node.js
compatibility, it is recommended that the main entry point is CommonJS...
I am finding that when I import a module that offers ES6 module format (via is "module" property in "package.json"), then CRA uses this as the entry point for the module.
Thus - even when my project is using CJS entirely, it then tries to import ES6 modules - and fails.
So - Is this a bug? Or, am I misunderstanding the intended behaviour of create-react-app?
Reproducing
I have reproduced this here: https://github.com/mjashanks/cra-test. This project is a basic CRA skeleton, modified to used CJS.
Additionally, I have added a "test-module", whose a package.json includes a "main" (CJS) and "module" (ES6) entry point.
If we run npm start, the project fails to build, as the file "test-module/index.esm.js" is being used. (We can edit this file to use CJS format to make the project build and make this more clear).
"test-module/index.cjs.js" is never used.
Thanks :)
I found that this actually turned out to be an issue (actually expected behaviour) with Webpack: https://github.com/webpack/webpack/issues/5756
I have a current project using aurelia.
I add the supertest library using npm.
npm install supertest --save-dev
Now add package to aurelia.json file
{
"name": "supertest",
"path": "../node_modules/supertest"
}
Now run aurelia build
au run
produces following error:
Tracing supertest...
error /Users/xxxxxxx/node_modules/supertest.js
Writing app-bundle.js...
I have tried everything I can think of to fix this. Any help would be appreciated.
I recommend you to use the aurelia-skeleton-navigation setup instead which uses Gulp and JSPM -> https://github.com/aurelia/skeleton-navigation. This will provide you more on control on your project.
Aurelia CLI is still on alpha, quoting from https://github.com/aurelia/cli:
Note: The CLI is currently in Alpha and as such may not be suitable for use on all projects yet. In particular, projects that need to make use of extensive 3rd party libraries or Aurelia plugins may not yet work or may require extensive custom configuration or workarounds. We are in the process of addressing these issues.
Emphasis on extensive custom configuration or workarounds lol. I myself tried the CLI initially but ended up switching to Gulp and JSPM setup instead because of the tedious importing of external libraries.
I'm trying to bundle my node.js application with webpack or browserify, but I need some backend modules such as knex, bookshelf and others.
But webpack and browserify fail to package these modules since they do some strange stuff with require()...
I got these kind of errors :Cannot find module 'sqlite3' or Error: Cannot find module './dialects/postgres/index.js'
I can't ignore my node_modules directory since I need the modules in the package because I can't access the environment where my package will be used. (AWS Lambda)
I don't need webpack or browserify to optimize my node_modules but I need them, is there a way to tell webpack or browserify to just bundle the node_module folder and trust me if a require is missing ?
EDIT: I'm using serverless to test and deploy my lambdas and the only plugins allowing me to use ES2015 with babel force me to use webpack / browserify
Thank you :)
You can incorporate node_modules in the your Lambda package (the zip file containing your code that you upload in Amazon Lambda) and don't need to package it (I mean create a file with Webpack or Browserify).
Some node modules are not meant to be used in the browser and do not support packaging because they use dynamic requires. They can have good reasons to do that, specially ORMs like Bookshelf or Sequelize.
Webpack can manage simple dynamic requires, but it works only for rules easy to parse. If you really want to use webpack for whatever reason, you could have a look to the ContextReplacementPlugin but I don't think it worth the effort.
I had this exact problem and finally got it to work with webpack (don't know about browserify) by adding this to my webpack config:
{
plugins: [
new webpack.NormalModuleReplacementPlugin(/\.\.\/migrate/, '../util/noop.js'),
new webpack.NormalModuleReplacementPlugin(/\.\.\/seed/, '../util/noop.js'),
new webpack.IgnorePlugin(/mariasql/, /\/knex\//),
new webpack.IgnorePlugin(/mssql/, /\/knex\//),
new webpack.IgnorePlugin(/mysql/, /\/knex\//),
new webpack.IgnorePlugin(/mysql2/, /\/knex\//),
new webpack.IgnorePlugin(/oracle/, /\/knex\//),
new webpack.IgnorePlugin(/oracledb/, /\/knex\//),
new webpack.IgnorePlugin(/pg-query-stream/, /\/knex\//),
new webpack.IgnorePlugin(/sqlite3/, /\/knex\//),
new webpack.IgnorePlugin(/strong-oracle/, /\/knex\//),
new webpack.IgnorePlugin(/pg-native/, /\/pg\//)
]
}
If you're using serverless-webpack like me, you'll have to explicitly npm install the normal webpack module in your project and require it in your webpack config file.
This config is specifically for my setup where I use postgres without pg-native. Just ignore the modules you're not using.
The two top plugins aren't needed for webpack to build, but they get rid of a ton of annoying warnings. They're probably a little dangerous since they could match requires in other modules than knex. I couldn't find a better way without having to write my own plugin.
I was struggling to get webpack to bundle knex properly and want to share my configuration.
We could instead use ContextReplacementPlugin to avoid listing all drivers not needed for the project as when using webpack.IgnorePlugin. A side benefit is that it would avoid ignoring these packages in other modules (as mention by #Erik Frisk).
For example, I use a mysql database with mysql2 driver and my webpack config looks like this:
plugins: [
new webpack.ContextReplacementPlugin(/knex\/lib\/dialects/, /mysql2\/index.js/),
]
This will only bundle node_modules/knex/lig/dialects/mysql2/index.js excluding other dialects thus effectively ignoring the dependency in other packages like oracledb, mssql, etc.
To find more information about ContextReplacementPlugin have a look at Webpack’s ContextReplacementPlugin examples:
I'm trying to figure out how to use external javascript libraries in the Atom editor. I used npm to install the momentjs library, since Atom uses node. However, I'm puzzled as to what to do now? I can't find a way to use the library in Atom.
I thought that I could go to the Atom init.coffee file and do a require "moment" (also tried require "momentjs") but nothing seems to work.
The whole reason behind this is so I can use some javascript libraries for formatting dates in a snippet (I have another SO question about that which I'll close if this solves it).
But this question is a general question about installing and running javascript libraries in Atom. I've looked through the Atom docs and Googled, but I can't find a good answer. I figured something like this would be pretty easy?
As Atom bundle its own node version (and thus is not using your global version(s)) it won't load globally installed modules through require.
However, the require method supporting absolute paths, you can still load any module if you know it's absolute path, which shouldn't be a problem in your specific case.
In your init script you can write:
momentjs = require('/path/to/momentjs')
But beware of modules that ships with binaries. Atom is using node 0.11.13 so if the module you're trying to require have been installed for a different version you'll get a Error: Module did not self-register.. In that case I'm afraid the only solution would be to install the module as a dependency of an Atom package (as suggested by #nwinkler).
You should be able to do the following when developing your own package:
Install moment using npm install --save moment - this will install the moment.js library as a dependency and register it in the package.json file
In your library, import it in your lib file:
moment = require 'moment';
myDate = moment().format();
Then you can use the moment object to format your timestamps.
All of this will only work if you're doing your own package, of course. Not sure if this will work with simple snippets as well.