I noticed that npm installs the same node.js packages several times when they are required by different modules.
Is there an option to switch this off and to have one repo for all the packages?
I looked but did not find anything on this...
This is the default behaviour of NPM since version 3. Quoting the changelog of that version:
Your dependencies will now be installed maximally flat. Insofar as is possible, all of your dependencies, and their dependencies, and THEIR dependencies will be installed in your project's node_modules folder with no nesting. You'll only see modules nested underneath one another when two (or more) modules have conflicting dependencies.
As you can see, if you have modules which have conflicting dependencies (i.e. require different versions of the same dependency), multiple versions of such dependencies will be installed, and this cannot be avoided.
If you suspect that your node_modules directory somehow contains duplicates, you can use the command npm dedupe (docs) to find these and reduce the duplication, leaving only one instance of each duplicate.
Related
We work in a large team on multiple projects and everyone has a different set of node packages installed globally.
This can cause different behaviour when different members of the team build a project.
Is there a setting in a package.json file (or other file) that we can apply that forces NPM to only use local packages that have been installed in the local project node_modules.
I assume that this is more relating to development tools and modules (netlify, standard, prettier, #babel/core, rollup, eslint, etc)
Using npm i --save-dev package-a package-b package-c to ensure that you have the exact and correct versions across your development teams. I would suggest looking at package.jsons in projects like three.js, react, nextjs etc to see how they handle it. It can be quite a task to get a correct baseline of devDependencies, but once set, it generally solves a lot of package dependency problems caused by global modules.
See this: http://npm.github.io/how-npm-works-docs/npm3/how-npm3-works.html
I have a monorepo where I have a /packages folder with many packages.
I can use npm i ./packages or npm i if they are already specified using using the file pointer.
Looks something like this:
"dependencies": {
"#reggi/command": "file:packages/command",
"#reggi/dep-merge": "file:packages/dep-merge",
"#reggi/dep-merge-cli": "file:packages/dep-merge-cli",
"#reggi/dep-pointer": "file:packages/dep-pointer"
}
The issue is that if I install these packages I don't get dev dependencies.
What I really want is to also install the devDependencies of these dependencies.
lerna a popular tool that has pioneered the usage of monorepos, suggests that you should add all the devDependencies for these packages in the root package. The issue with this is that it eliminates the ability for two packages to depend on different versions of a given dev dependency.
What I have done is created a script that merges all the devDependencies into dependencies at preinstall then undoes the changes. This works but can be kind of wonky at times, especially when explaining all this to shrinkwrap.
It would be nice if I could just npm i --allDevDepsFromDeps and it would install all of my dependencies dev dependencies.
Is there any other solution I am missing?
I don't see what you're trying to achieve there, aren't the devDepencies used for ... development?
If you want different version for different package just don't put them in the root but in each package.
The issue is that if I install these packages I don't get dev dependencies.
You should consider those packages as 'production'/'bundled' packages, you don't need dev dependencies in this case.
For example, when you are working on #pkg/A, it will have its own devDep but then if you work on #pkg/B that depends on #pkg/A, the #pkg/A should be the production/bundled version (without devDeps).
Maybe you should have a look at bundledDependencies or peerDependencies, that might help you.
On my current project we have dependencies listed with the exact version (e. g. babel-core: '6.5.2') in our package.json file.
But after the recent project build (which includes running npm install on the build machine) some node modules were updated and our artifact got broken.
The research has shown that although we have the exact versions mentioned in package.json file, the required module might have dependency on itself through its dependency, and both of them are using the caret dependencies. In our case it appeared to be
babel-core: '6.5.2' depending on
babel-register: '^6.16.0' depending on
babel-core: '^6.9.0' resolved as babel-core: '6.26.0' (because of the caret range rules)
And the specified babel-core version breaks our builds.
We've tried creating package-lock.json files in order to prevent these transitive dependencies to be resolved the way they were resolved.
But, as the documentation on package-lock files states:
Whenever you run npm install, npm generates or updates your package lock
And thus, all the packages depending on babel-core with caret version are resolved with incorrect version of a module.
After taking a closer look at package-lock.json file, I don't see it's worth changing it because of
the snowball effect - changing one dependency requires to change all the others, depending on it (and this is recursive process)
the need of verifying both the existence of the version you're about to change to and its hash (for the integrity field)
The question is: how do I install the exact versions of all the modules, mentioned in my package.json file, preventing NPM to resolve it again with a different version?
Try using npm shrinkwrap, it is used for transitive lock.
I am new to nodejs. And I found that there are many unknown modules in node_modules folder after I installed three modules (express, jade, gulp) in my local project.
Unknown module examples in the node_modules:
vary
statuses
send
promise
From the tutorials I see from others, after they installed gulp, there will be only one "gulp" folder in their node_modules folder, but this is not my case. Why? Thank you.
The node package manager (npm) updated recently, as part of that update, all modules are installed in the top level node_modules folder. This includes modules that your dependencies need to install. In the past these modules would be nested inside another node_modules folder in express (for example).
This is why the tutorials you read say different, likely they were written before this update.
It is mentioned in the npm changelog here
Your dependencies will now be installed maximally flat. Insofar as is
possible, all of your dependencies, and their dependencies, and THEIR
dependencies will be installed in your project's node_modules folder
with no nesting. You'll only see modules nested underneath one another
when two (or more) modules have conflicting dependencies.
#3697 This will hopefully eliminate most cases where windows users ended up with paths that were too long for Explorer and other standard
tools to deal with.
#6912 (#4761 #4037) This also means that your installs will be deduped from the start.
#5827 This deduping even extends to git deps.
#6936 (#5698) Various commands are dedupe aware now. This has some implications for the behavior of other commands:
npm uninstall removes any dependencies of the module that you
specified that aren't required by any other module. Previously, it
would only remove those that happened to be installed under it,
resulting in left over cruft if you'd ever deduped. npm ls now shows
you your dependency tree organized around what requires what, rather
than where those modules are on disk.
#6937 npm dedupe now flattens the tree in addition to deduping. And bundling of dependencies when packing or publishing changes too:
#2442 bundledDependencies no longer requires that you specify deduped sub deps. npm can now see that a dependency is required by something
bundled and automatically include it. To put that another way,
bundledDependencies should ONLY include things that you included in
dependencies, optionalDependencies or devDependencies.
#5437 When bundling a dependency that's both a devDependency and the child of a regular dependency, npm bundles the child dependency. As a
demonstration of our confidence in our own work, npm's own
dependencies are now flattened, deduped, and bundled in the npm#3
style. This means that npm#3 can't be packed or published by npm#2,
which is something to be aware of if you're hacking on npm.
I just run a simple npm install morgan in a folder and for some reason it adds all the sub dependencies to the parent folder. See image attached
Yes, this is a new feature in npm 3.x, you can read about it here:
https://github.com/npm/npm/releases/tag/v3.0.0
Your dependencies will now be installed maximally flat. Insofar as is possible, all of your dependencies, and their dependencies, and THEIR dependencies will be installed in your project's node_modules folder with no nesting. You'll only see modules nested underneath one another when two (or more) modules have conflicting dependencies.
Basically, it now handles dependencies with as little nesting as possible.
Npm has changed the way they organize dependencies. So instead of 2 separate modules requiring the same dependency and installing them in their own node_modules folder. The dependency is only installed once at the same folder level the node module is installed at.